From patchwork Sat Feb 8 09:02:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolin Chen X-Patchwork-Id: 863502 Received: from NAM04-MW2-obe.outbound.protection.outlook.com (mail-mw2nam04on2041.outbound.protection.outlook.com [40.107.101.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D3341A00F2; Sat, 8 Feb 2025 09:03:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.101.41 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739005404; cv=fail; b=jXdiO5DrzzadnzJAENbPS8ajvI1WAZWTT+0AY9KFQfj8ZxbPYFr5LQeIjwm5/We47abmdaAqxPaJFO6Zt5e3WktjrEkCuCotXcgcso2A4cMw1uURTOuCPGk9L0PQdrSSwoSCe9X0u+ooo1t1nC3xW4eM3xSsrqYlAibseCrlMwY= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739005404; c=relaxed/simple; bh=FheWkBMhjruQgnxgg0RyItMCXEO72CKXmE09MXP65SU=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=TMDfxoBx5+OK9EkrtNXJ5ha2MYmykDfSrGhDTYeBhhDAkRoB/sO5abPTlaciAIj99akbr69YGZ/431aUP78d5tJd1QObPMksc3hLsQ+kLWM5CJXhjYB+2Nw1tV9F3Jki9y5+tRJ2gwrTn1x3y4FqlHXKVCq6B1QTYgYD1cSLr6o= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=VVK+vxx0; arc=fail smtp.client-ip=40.107.101.41 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="VVK+vxx0" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=GNo5xzPHbnX7YUZ1A8oLyK/fBSjEqG+v6KR8iQ88T+f2UzXgtCQ/qOhZV6XiwIQezm7z7RMJrMQWTpyX23GRJ4JZBNLqldmCqaUpesGInAOpy2BKTf7on8vO0y1ZHgLk5lzE+G1uQ94tHnM+2LrmzKV+800eTcsaKN+bXtaneJbLo/nLj+0NFZ6xHRk7xa+SM5AevcACpS8kQKOT1lsG/hN8f7wmQ5sDtww9N0upHrUjbeBnHDWfKXCTqSx2KT7t71bnsjMfgUqkLWkgAyQtb+jSMgrpCz7H6H0XIh2169399KXeOD2QMs2ScFkN5647k+rVAZsl0MaI+VRQkr3bPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=U/AmL8qaAGihw4P5IwftJNsywAVlmmaEhofPXsynnuY=; b=fWu8tbzl9zxrUkJmsThqQTT9FQI7snMXdlyukXNgJOk2Dhll8hig833ktPPpQcs9WlZTkTzYi1QbmiW/9VwoERGkW8OuMbUyGiY9/cKVCdNHVLMrfDm5wLoJI8g6kOcmwGwVnngA8AN0GeiYqMKPYnWg1AwBVbuWmyRmLdbD4pBQlrN9PTn8dIvZKKblJ9utYCCM6Byl2stOTon3Ygpij1fLyqZUahon2CLVJVzT3HCzTXoYt3QZaIAGq4VYRChw1j7+2fI0gegwocGYwXMLGQY28mcvDBAQ1e/Lobs4IQjeM3lRIgoITftf13relg87z/caAAfRzBeLNSS1XUUm2A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=gmail.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=U/AmL8qaAGihw4P5IwftJNsywAVlmmaEhofPXsynnuY=; b=VVK+vxx0kaWlc+8WcjEQVm0qYXb4js429suBLNT8B1d7DFtIZsngxZmUSb7gaoAMpXXcgMbcFBywjWxCflucvmTSRaD7SCHCo0h6rppXx4FlPPIjNVrqhmFlnWQ1gRbrqFZiu2npj0Bb9wSmi2bsM/4nDYZ2a98PxqWSLbstasuLRpMPw4wdQLD4nL7uwjP/1RLeGghPfSD4SvS1K9ja9cib5s0vgSWZOdS11FEAcBA3LBZGjInKfDPPqDjsoAb8jDLVhw33UvuSY/dLvIjVzgmx0nQOs+Pz13CXKVceJx6X111WiN5fDINmdd7p0nAMj6OFy2aQoMrpXI0Vzl/K9w== Received: from DS7PR05CA0021.namprd05.prod.outlook.com (2603:10b6:5:3b9::26) by DS0PR12MB8503.namprd12.prod.outlook.com (2603:10b6:8:15a::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8422.16; Sat, 8 Feb 2025 09:03:17 +0000 Received: from DS3PEPF000099D5.namprd04.prod.outlook.com (2603:10b6:5:3b9:cafe::f2) by DS7PR05CA0021.outlook.office365.com (2603:10b6:5:3b9::26) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8445.6 via Frontend Transport; Sat, 8 Feb 2025 09:03:17 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DS3PEPF000099D5.mail.protection.outlook.com (10.167.17.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8398.14 via Frontend Transport; Sat, 8 Feb 2025 09:03:17 +0000 Received: from rnnvmail205.nvidia.com (10.129.68.10) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sat, 8 Feb 2025 01:03:08 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail205.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sat, 8 Feb 2025 01:03:08 -0800 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Sat, 8 Feb 2025 01:03:07 -0800 From: Nicolin Chen To: , , , CC: , , , , , , , , , , , , , Subject: [PATCH v1 01/13] genirq/msi: Store the IOMMU IOVA directly in msi_desc instead of iommu_cookie Date: Sat, 8 Feb 2025 01:02:34 -0800 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF000099D5:EE_|DS0PR12MB8503:EE_ X-MS-Office365-Filtering-Correlation-Id: b7aa029d-703c-446c-ef15-08dd481f6d9e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|7416014|82310400026|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: PndUM65KHMprDwWHaCUjOyGs8hSf+37qo7tSlkXItXp0xmPUTYQ61uif+LLk8lgXUc0KN2lCP5M4zkwW33XBgzHd/apv8i8JZqAgSYhSFWq0ffuLPLWxoK37G0B2v/IaKL2zlfEh1uBIg/A0ylPXh+l3TTxje0HlGDJF6qzif42M0wf850Ld3rzHGm/NUjkD+HZP83ujZIsFR5fX2PwuoDbpBFoPYuVJiHk6lFSpg/gtT45JNcVBkzar+h8AeMEXlOHk+rXnlMwmenpjwGXKb6H6+komkfoKl9D8L1WA4Km0mpc+DM28GMoiJdOSiQ2czWUPu9iN52f88FvBjfAhmupxDaS5HdPz12ZfPN/gzuAKiNC/6+QNgvRouzNYPEozuU0HaLVjOi9QiAudbjiXk+0n6AAQca/BthtVFs89BKxsaqfsZ0arA0VmItgI3xmn+p0uew2IHoGP0C5fTt3tHnbHuZJk5QkBFHjln8v0xmvQ+8nYRrLSq4GeMeHWeKvf0rcX0f71EJiaKIGywxxVE+Son1GZEv2BpU/P9jNHZLJyIBaIt3ADBw0LP7YveRHCi3DzV7UVAT0PVuYdn6yCzZBhO8dmWrE4DrXZYLbKz991WDkj/RGl+mGxGYPjU0tvjYyqTA9vX0ynUHm0oq/dUp1pObzgmj+czxwaFlniLYl0SebLsVck4ltUjt0gbY0U7EKKiQK7jtgG7TS98QYB/xpX58Y+UePR5jiSLxopo5j0WWGGLg9RQBYmEcrhrwxJoAnPfG2cw5NCOSB0F7a0U1BX8KafqLRkEa2cPtqpb49uuySVcHAIF4n/rUG+tEIRRD1Aoo9XlHuBepeSHJaPc70FK36yw523iSMH2LEUP+U9Bu5rc7XBA9m6I3mi2G/b5lljwNxXjGDeHhldXZk7axtl6enamSdqLdY1CiNtBHyfFbEzORU5iRBF0UmaBmUpavEbxMPOQog9tM7bbI2BsMaj4vQeieDWRDFLWHNmLhVDtPc21vBi6dGB01BXJuu0LIk8kYl/Vid64WAxESmDveLfwbUBsCKtr2WnzonYHsW5qjUWVGLgLvETID4AZ5MYjlQP+rnu0Gs7tQlPb0Ki+GvUvQG/2bz/3cTFNDPt5WxGQhGEUzlgz4u/W25/UL+AucU4SVxpoX0DkqRyxCNNIzSsz8Fa2RkaEcPT4uoQRqK60HCTbrzRCr4RJYqyV3+MF/pF2Wb/O5JDO5gIQZsRgGolJ+LlG/hxgNvNi67xFz7vgzXB08jTaX+6hmi+FaHsf6zA/We6SxNmMXDtU7TAo86YMBARbInKALrugsZl18EXITE7Hqj3L5EQfZuslCa/3JQdcRVfmtqVUB1E/CEwnZ933rVLKp9U6hHUbagHBdJjlF5QhEgAVEYzgOGG+ltE5EPdKWWMZEvTg76PalF9Es0lEG3DSDB1OQVJB9UbulgeCGk1D3QfoR5iNzLywPOtDj4kk8K5PWFenclLygrHzTqDXPZs8vmTt0UTASuHP80= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(376014)(7416014)(82310400026)(36860700013)(1800799024); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Feb 2025 09:03:17.5390 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b7aa029d-703c-446c-ef15-08dd481f6d9e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF000099D5.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8503 From: Jason Gunthorpe All the iommu cases simply want to override the MSI page's address with the IOVA that was mapped through the iommu. This doesn't need a cookie pointer, we just need to store the IOVA and its page size in the msi_desc. Instead provide msi_desc_set_iommu_msi_iova() which allows the IOMMU side to specify the IOVA that the MSI page is placed during iommu_dma_prepare_msi(). This is stored in the msi_desc and then iommu_dma_compose_msi_msg() is a simple inline that sets address_hi/lo. The next patch will correct the naming. This is done because we cannot correctly lock access to group->domain in the atomic context that iommu_dma_compose_msi_msg() is called under. Today the locking miss is tolerable because dma_iommu.c operates under an assumption that the domain does not change while a driver is probed. However iommufd now permits the domain to change while the driver is probed and VFIO userspace can create races with IRQ changes calling iommu_dma_prepare_msi/compose_msi_msg() and changing/freeing the iommu_domain. Removing the pointer, and critically, the call to iommu_get_domain_for_dev() during compose resolves this race. Signed-off-by: Jason Gunthorpe Signed-off-by: Nicolin Chen --- include/linux/iommu.h | 6 ----- include/linux/msi.h | 49 +++++++++++++++++++++++++-------------- drivers/iommu/dma-iommu.c | 30 ++++-------------------- 3 files changed, 36 insertions(+), 49 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 38c65e92ecd0..caee952febd4 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -1508,7 +1508,6 @@ static inline void iommu_debugfs_setup(void) {} int iommu_get_msi_cookie(struct iommu_domain *domain, dma_addr_t base); int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_addr); -void iommu_dma_compose_msi_msg(struct msi_desc *desc, struct msi_msg *msg); #else /* CONFIG_IOMMU_DMA */ @@ -1524,11 +1523,6 @@ static inline int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_a { return 0; } - -static inline void iommu_dma_compose_msi_msg(struct msi_desc *desc, struct msi_msg *msg) -{ -} - #endif /* CONFIG_IOMMU_DMA */ /* diff --git a/include/linux/msi.h b/include/linux/msi.h index b10093c4d00e..74c6a823f157 100644 --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -166,6 +166,10 @@ struct msi_desc_data { * @dev: Pointer to the device which uses this descriptor * @msg: The last set MSI message cached for reuse * @affinity: Optional pointer to a cpu affinity mask for this descriptor + * @iommu_msi_iova: Optional IOVA from the IOMMU to override the msi_addr. + * Only used if iommu_msi_page_shift != 0 + * @iommu_msi_page_shift: Indicates how many bits of the original address + * should be preserved when using iommu_msi_iova. * @sysfs_attr: Pointer to sysfs device attribute * * @write_msi_msg: Callback that may be called when the MSI message @@ -184,7 +188,8 @@ struct msi_desc { struct msi_msg msg; struct irq_affinity_desc *affinity; #ifdef CONFIG_IRQ_MSI_IOMMU - const void *iommu_cookie; + u64 iommu_msi_iova : 58; + u64 iommu_msi_page_shift : 6; #endif #ifdef CONFIG_SYSFS struct device_attribute *sysfs_attrs; @@ -285,28 +290,36 @@ struct msi_desc *msi_next_desc(struct device *dev, unsigned int domid, #define msi_desc_to_dev(desc) ((desc)->dev) -#ifdef CONFIG_IRQ_MSI_IOMMU -static inline const void *msi_desc_get_iommu_cookie(struct msi_desc *desc) -{ - return desc->iommu_cookie; -} - -static inline void msi_desc_set_iommu_cookie(struct msi_desc *desc, - const void *iommu_cookie) +static inline void msi_desc_set_iommu_msi_iova(struct msi_desc *desc, + u64 msi_iova, + unsigned int page_shift) { - desc->iommu_cookie = iommu_cookie; -} -#else -static inline const void *msi_desc_get_iommu_cookie(struct msi_desc *desc) -{ - return NULL; +#ifdef CONFIG_IRQ_MSI_IOMMU + desc->iommu_msi_iova = msi_iova >> page_shift; + desc->iommu_msi_page_shift = page_shift; +#endif } -static inline void msi_desc_set_iommu_cookie(struct msi_desc *desc, - const void *iommu_cookie) +/** + * iommu_dma_compose_msi_msg() - Apply translation to an MSI message + * @desc: MSI descriptor prepared by iommu_dma_prepare_msi() + * @msg: MSI message containing target physical address + */ +static inline void iommu_dma_compose_msi_msg(struct msi_desc *desc, + struct msi_msg *msg) { -} +#ifdef CONFIG_IRQ_MSI_IOMMU + if (desc->iommu_msi_page_shift) { + u64 msi_iova = desc->iommu_msi_iova + << desc->iommu_msi_page_shift; + + msg->address_hi = upper_32_bits(msi_iova); + msg->address_lo = lower_32_bits(msi_iova) | + (msg->address_lo & + ((1 << desc->iommu_msi_page_shift) - 1)); + } #endif +} int msi_domain_insert_msi_desc(struct device *dev, unsigned int domid, struct msi_desc *init_desc); diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 2a9fa0c8cc00..bf91e014d179 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -1815,7 +1815,7 @@ int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_addr) static DEFINE_MUTEX(msi_prepare_lock); /* see below */ if (!domain || !domain->iova_cookie) { - desc->iommu_cookie = NULL; + msi_desc_set_iommu_msi_iova(desc, 0, 0); return 0; } @@ -1827,33 +1827,13 @@ int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_addr) mutex_lock(&msi_prepare_lock); msi_page = iommu_dma_get_msi_page(dev, msi_addr, domain); mutex_unlock(&msi_prepare_lock); - - msi_desc_set_iommu_cookie(desc, msi_page); - if (!msi_page) return -ENOMEM; - return 0; -} -/** - * iommu_dma_compose_msi_msg() - Apply translation to an MSI message - * @desc: MSI descriptor prepared by iommu_dma_prepare_msi() - * @msg: MSI message containing target physical address - */ -void iommu_dma_compose_msi_msg(struct msi_desc *desc, struct msi_msg *msg) -{ - struct device *dev = msi_desc_to_dev(desc); - const struct iommu_domain *domain = iommu_get_domain_for_dev(dev); - const struct iommu_dma_msi_page *msi_page; - - msi_page = msi_desc_get_iommu_cookie(desc); - - if (!domain || !domain->iova_cookie || WARN_ON(!msi_page)) - return; - - msg->address_hi = upper_32_bits(msi_page->iova); - msg->address_lo &= cookie_msi_granule(domain->iova_cookie) - 1; - msg->address_lo += lower_32_bits(msi_page->iova); + msi_desc_set_iommu_msi_iova( + desc, msi_page->iova, + ilog2(cookie_msi_granule(domain->iova_cookie))); + return 0; } static int iommu_dma_init(void) From patchwork Sat Feb 8 09:02:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolin Chen X-Patchwork-Id: 863500 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2074.outbound.protection.outlook.com [40.107.92.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 98D521B21B2; Sat, 8 Feb 2025 09:03:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.92.74 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739005410; cv=fail; b=St+LE6d/24y4An4TpFvupemCVoCugWTMZLYyUGvAWnJsU9Cgg7v+rqqd6rnn4uLclSEb2UM9k0YOsAQF6jZ7Qp8bSrzkyHN4HxhZsvU5ka9ySM3dLUbjWYUC4pTUtbdfE9y4eOQ1AIHmkzP+BxF5TwlYH8pGak7nHMHB+UeRW8s= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739005410; c=relaxed/simple; bh=j/3lCswW8XRUuMz13KYbup7ioGDwHt+Ti1U4Zcevxr0=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Iup4WKVrRf7XGf/zZljtHwXQGsO9DchokatRE1KT1HdjerakLT4etOBixEWeiUxI/PxB8pqtCSDiDbfYItHbelKm27/i3MRC/4vBQJpWJTc1MT71p6sL31tv0OT0v/arY/HC/0swU9z+irasEhRYgNGYPG1ovM4ahSwhwRfcsKE= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=thtG0tOY; arc=fail smtp.client-ip=40.107.92.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="thtG0tOY" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=oB6doFU6sfNk65eAovo6i8pacQm5RAucXS3fUlvS89ayWkPgKHR57l89+g4kxW9nmYkXjoTHm31T6m85wpvwD91ZzjJo9e8x9X32ci/Lox0F+XV4qYQxCVLlBsGsZlYGU6dRlYtZwrj5iTK1J5CXG+KAnE9I26HuGtvQvkVBHhSlTOFTIuUk5r+QpRU1xI0n9P21Ozsz2Y0EJtKkWzgzG92Y4Qrgubzyo/YReVMHHz4SD82+0WYGbHoC90qke8OcgABRqwvyeUfQP24P9P1HJSRzBPQWR1UQv0eAntNXOH6EjE7Ovojg6DcZrJxP8LM4LIw2bCSWA1WMnkoWTd32XQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=bhPOTuzja0Y6NxE5hxlX7yg52nGBhP1A/JjD9CRO/rc=; b=e3Tefk9cmaDAPh5lJKDY6UTmf1cA2CRed62sqVew5YZnsTPqizSwTCUh8SJf2Pm57k7XOR8HEbsiMXrAyMSWe/1XLPYqOUoJAwdD7XcetgeqMXLfyFi3HnuWfFFpgIh89biR/jAmraqyfboHVpJwq4aq2OxTvovJ2YIGwDXXzbHCN9f5ZuLivqn5+n7hZ4NlpoCx1JYIV0EIWUhNxZeqlkJWyQ9J5AMXXtVwxRNm3tBOGxG+CLWvwjudYXP5bi1E/yPGH3umNDeTFjtzj+L0zyccK5lbl6aYdczKvi/39h54mr7P/FcE1+89ru/bwWcW5juB35lQQcy/QFIDAi1F4A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=gmail.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=bhPOTuzja0Y6NxE5hxlX7yg52nGBhP1A/JjD9CRO/rc=; b=thtG0tOYTCDeJByV/PGbVzyroDJDYGini2SgQjwCRihisZBnwrb4Sq0+nOY2UPcaHPc01GmoHopM9lbj8Ph9N10ME2MWy3hC607XJQzwIM+PpneJWRtnRqLYrrYZ7P5U13gXmBNRWFnAa0ixtCrf0j4P+cHgLXC9CiW2rAudQrdq+IaV6DL1HjZ4wW8hDR0pNyMPKg7lp6KKtdPBNSSuTOANgKAI7iy41MukcXyrbBw9pQj5jhhOBlGxiXNBeJ0TezOdRGrKfWmJkm8X8fsC+294CSRAR/QQZ4D0IUy8RoGE2eaHU4uA6YRpMqdAyBvVeN5dzd2qbm2TuN1IRXbodA== Received: from BY3PR03CA0017.namprd03.prod.outlook.com (2603:10b6:a03:39a::22) by CH3PR12MB7545.namprd12.prod.outlook.com (2603:10b6:610:146::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8422.11; Sat, 8 Feb 2025 09:03:22 +0000 Received: from SJ1PEPF000026C8.namprd04.prod.outlook.com (2603:10b6:a03:39a:cafe::a7) by BY3PR03CA0017.outlook.office365.com (2603:10b6:a03:39a::22) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8398.30 via Frontend Transport; Sat, 8 Feb 2025 09:03:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by SJ1PEPF000026C8.mail.protection.outlook.com (10.167.244.105) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8398.14 via Frontend Transport; Sat, 8 Feb 2025 09:03:22 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sat, 8 Feb 2025 01:03:10 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sat, 8 Feb 2025 01:03:10 -0800 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Sat, 8 Feb 2025 01:03:09 -0800 From: Nicolin Chen To: , , , CC: , , , , , , , , , , , , , Subject: [PATCH v1 03/13] iommu: Make iommu_dma_prepare_msi() into a generic operation Date: Sat, 8 Feb 2025 01:02:36 -0800 Message-ID: <432bc128b36b9dd57329b203bf46e01683a32f06.1739005085.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF000026C8:EE_|CH3PR12MB7545:EE_ X-MS-Office365-Filtering-Correlation-Id: c7501fcd-c58b-4dec-40d2-08dd481f7049 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|7416014|376014|82310400026|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: /B9tFG7sTFvukp4CLi/gJtCxArhdl4WaeZ5hdnvcZIbgCvte/tQoJ9BL825Ex5MPaEcvesiTbdyU8A9B0LbU8wZhv5AVLWGw1SBL/ugNseKc+w4LOTwJWzBZkrHAKZt4rN7CMO9CL73YOOuewIrBGDqz0th5UHFQWwqRsQvxxlsSSJOYTDVveXPY5EPRkqF/02jHbaL/WrswEMOjl0oRQ2dIe7Sz/WmhCnN9iwj/8TezE1PlXPSFIYXdUYntN9vAXZEv568fC4LeuQ23LDDM+M8h9JwMOfaYsX9m4AyAyjNTzK5fFqTp8XuNW9ni29+90sUfxTHJzCCQ1boXhlYMMzpF/0lmUX0+4JPytjDe81PEOa2dWwmvAf9XkriUU5DG7lu2M4vKcWv20y9dopIyZxMeOHFflhiwSJOQRvhZiLN9Ccl8Zx1rdkNKldJQqZsGgayuv1uXfIm8EUD5ojw2ajpnFyoNmHDzG/xEwkTlSv/u5aS62WkZSm5nmedAauBKQpU/JqSU9gAqktY3Fbm3hMgOBpufetCQHpp+L7tmCTvMSDYXnPhSd3/JosadJO8ik9thFtHhZtZ/fPYr1vXnSUeLydnFMzXsTXEb/8iJ2Dv40W1QkyRNSF95DDe4u0FBj4VoE8L5TLzeKdo8Fg/1ONH7BuB/dBQJKWf6yH2nZDJxO7GWgnnpE5voPTmow7l/oCdLKu1nUiBJIEa0p49S8mb8GaSutuNRtyA5BlbqEiTOhV/YCTLHEH/rkxzUSswb0Zae/nLFVlFvlRDMF8vLPKJ8KTuznOB3Kj/wX5uexxrhcL0Yuh4FLSS4jSjYc3DT09DjI9TxdCAN593Rpyn8vAeGxJA39nsOhDqGoSaI1vpEaySGMUt03DtsO/2ouY47uV6QvzjQ0ljKCdxik1pvHodanoYINPtj4Tdmu9F7kYSfoP1QdYJbjAaMAdPRT39fUBHj+jkSudGzwxIqi4VcPdsCC0GfXSdr6UPD+TmkUIxGRDzqqgaQbrZhF0SGhn/NeBhOuPNAbzw7OlBmmTJIgziULvVfou5t1f0iU8Jg0ecWuAIKqiXDiWAd1sI7UGEOMwdl8XCs67NgFL9VLN68DroV1TiECC6PioUvbzoN7vxUNWJEVKoJ7knDx4yWle9rGngtYLmSDjgmWAakVc5mE9c6dUxJ2PP2KE7qh/BAvQCY+iobgJNtDnJGwYUuEHkm12kxQDV93NT5QJC4rkWgOVmKNkJnFO6MU9pbcn3Hwz4x5aPWKXjBCVUZMBI8YOLjnNXz08XFyT2xJICDXS5Op66A+7ncBU478VvT0mSaK5Y73uqGNefbNUa3VbFTm6GJ9pJMiEdMZRF8rgh1IpXzy//gT7hmhi/GDSJTOWVk67+w+FGiHF1JaObjVRnRYjcOwdyaHQRZxFA48YtzVFTReZNSLoTYd9eZD5uplDRFNYcIO6J1wN6H95Uzyl/rJ/DYSzMU/uxnKwUZsjzdqBlQB+L/6WUGH9rauo07iHjHLpo= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(7416014)(376014)(82310400026)(36860700013)(1800799024); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Feb 2025 09:03:22.0262 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c7501fcd-c58b-4dec-40d2-08dd481f7049 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF000026C8.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB7545 From: Jason Gunthorpe SW_MSI supports IOMMU to translate an MSI message before the MSI message is delivered to the interrupt controller. On such systems the iommu_domain must have a translation for the MSI message for interrupts to work. The IRQ subsystem will call into IOMMU to request that a physical page be setup to receive MSI message, and the IOMMU then sets an IOVA that maps to that physical page. Ultimately the IOVA is programmed into the device via the msi_msg. Generalize this to allow the iommu_domain owner to provide its own implementation of this mapping. Add a function pointer to struct iommu_domain to allow the domain owner to provide an implementation. Have dma-iommu supply its implementation for IOMMU_DOMAIN_DMA types during the iommu_get_dma_cookie() path. For IOMMU_DOMAIN_UNMANAGED types used by VFIO (and iommufd for now), have the same iommu_dma_sw_msi set as well in the iommu_get_msi_cookie() path. Hold the group mutex while in iommu_dma_prepare_msi() to ensure the domain doesn't change or become freed while running. Races with IRQ operations from VFIO and domain changes from iommufd are possible here. Replace the msi_prepare_lock with a lockdep assertion for the group mutex as documentation. For the dmau_iommu.c each iommu_domain is unique to a group. Signed-off-by: Jason Gunthorpe [nicolinc: move iommu_domain_set_sw_msi() from iommu_dma_init_domain() to iommu_dma_init_domain(); add in iommu_put_dma_cookie() an sw_msi test] Signed-off-by: Nicolin Chen --- include/linux/iommu.h | 44 ++++++++++++++++++++++++++------------- drivers/iommu/dma-iommu.c | 33 +++++++++++++---------------- drivers/iommu/iommu.c | 29 ++++++++++++++++++++++++++ 3 files changed, 73 insertions(+), 33 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index caee952febd4..761c5e186de9 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -44,6 +44,8 @@ struct iommu_dma_cookie; struct iommu_fault_param; struct iommufd_ctx; struct iommufd_viommu; +struct msi_desc; +struct msi_msg; #define IOMMU_FAULT_PERM_READ (1 << 0) /* read */ #define IOMMU_FAULT_PERM_WRITE (1 << 1) /* write */ @@ -216,6 +218,12 @@ struct iommu_domain { struct iommu_domain_geometry geometry; struct iommu_dma_cookie *iova_cookie; int (*iopf_handler)(struct iopf_group *group); + +#if IS_ENABLED(CONFIG_IRQ_MSI_IOMMU) + int (*sw_msi)(struct iommu_domain *domain, struct msi_desc *desc, + phys_addr_t msi_addr); +#endif + void *fault_data; union { struct { @@ -234,6 +242,16 @@ struct iommu_domain { }; }; +static inline void iommu_domain_set_sw_msi( + struct iommu_domain *domain, + int (*sw_msi)(struct iommu_domain *domain, struct msi_desc *desc, + phys_addr_t msi_addr)) +{ +#if IS_ENABLED(CONFIG_IRQ_MSI_IOMMU) + domain->sw_msi = sw_msi; +#endif +} + static inline bool iommu_is_dma_domain(struct iommu_domain *domain) { return domain->type & __IOMMU_DOMAIN_DMA_API; @@ -1470,6 +1488,18 @@ static inline ioasid_t iommu_alloc_global_pasid(struct device *dev) static inline void iommu_free_global_pasid(ioasid_t pasid) {} #endif /* CONFIG_IOMMU_API */ +#ifdef CONFIG_IRQ_MSI_IOMMU +#ifdef CONFIG_IOMMU_API +int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_addr); +#else +static inline int iommu_dma_prepare_msi(struct msi_desc *desc, + phys_addr_t msi_addr) +{ + return 0; +} +#endif /* CONFIG_IOMMU_API */ +#endif /* CONFIG_IRQ_MSI_IOMMU */ + #if IS_ENABLED(CONFIG_LOCKDEP) && IS_ENABLED(CONFIG_IOMMU_API) void iommu_group_mutex_assert(struct device *dev); #else @@ -1503,26 +1533,12 @@ static inline void iommu_debugfs_setup(void) {} #endif #ifdef CONFIG_IOMMU_DMA -#include - int iommu_get_msi_cookie(struct iommu_domain *domain, dma_addr_t base); - -int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_addr); - #else /* CONFIG_IOMMU_DMA */ - -struct msi_desc; -struct msi_msg; - static inline int iommu_get_msi_cookie(struct iommu_domain *domain, dma_addr_t base) { return -ENODEV; } - -static inline int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_addr) -{ - return 0; -} #endif /* CONFIG_IOMMU_DMA */ /* diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index bf91e014d179..3b58244e6344 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -24,6 +24,7 @@ #include #include #include +#include #include #include #include @@ -102,6 +103,9 @@ static int __init iommu_dma_forcedac_setup(char *str) } early_param("iommu.forcedac", iommu_dma_forcedac_setup); +static int iommu_dma_sw_msi(struct iommu_domain *domain, struct msi_desc *desc, + phys_addr_t msi_addr); + /* Number of entries per flush queue */ #define IOVA_DEFAULT_FQ_SIZE 256 #define IOVA_SINGLE_FQ_SIZE 32768 @@ -398,6 +402,7 @@ int iommu_get_dma_cookie(struct iommu_domain *domain) return -ENOMEM; mutex_init(&domain->iova_cookie->mutex); + iommu_domain_set_sw_msi(domain, iommu_dma_sw_msi); return 0; } @@ -429,6 +434,7 @@ int iommu_get_msi_cookie(struct iommu_domain *domain, dma_addr_t base) cookie->msi_iova = base; domain->iova_cookie = cookie; + iommu_domain_set_sw_msi(domain, iommu_dma_sw_msi); return 0; } EXPORT_SYMBOL(iommu_get_msi_cookie); @@ -443,6 +449,9 @@ void iommu_put_dma_cookie(struct iommu_domain *domain) struct iommu_dma_cookie *cookie = domain->iova_cookie; struct iommu_dma_msi_page *msi, *tmp; + if (domain->sw_msi != iommu_dma_sw_msi) + return; + if (!cookie) return; @@ -1800,33 +1809,19 @@ static struct iommu_dma_msi_page *iommu_dma_get_msi_page(struct device *dev, return NULL; } -/** - * iommu_dma_prepare_msi() - Map the MSI page in the IOMMU domain - * @desc: MSI descriptor, will store the MSI page - * @msi_addr: MSI target address to be mapped - * - * Return: 0 on success or negative error code if the mapping failed. - */ -int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_addr) +static int iommu_dma_sw_msi(struct iommu_domain *domain, struct msi_desc *desc, + phys_addr_t msi_addr) { struct device *dev = msi_desc_to_dev(desc); - struct iommu_domain *domain = iommu_get_domain_for_dev(dev); - struct iommu_dma_msi_page *msi_page; - static DEFINE_MUTEX(msi_prepare_lock); /* see below */ + const struct iommu_dma_msi_page *msi_page; - if (!domain || !domain->iova_cookie) { + if (!domain->iova_cookie) { msi_desc_set_iommu_msi_iova(desc, 0, 0); return 0; } - /* - * In fact the whole prepare operation should already be serialised by - * irq_domain_mutex further up the callchain, but that's pretty subtle - * on its own, so consider this locking as failsafe documentation... - */ - mutex_lock(&msi_prepare_lock); + iommu_group_mutex_assert(dev); msi_page = iommu_dma_get_msi_page(dev, msi_addr, domain); - mutex_unlock(&msi_prepare_lock); if (!msi_page) return -ENOMEM; diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 870c3cdbd0f6..022bf96a18c5 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -3596,3 +3596,32 @@ int iommu_replace_group_handle(struct iommu_group *group, return ret; } EXPORT_SYMBOL_NS_GPL(iommu_replace_group_handle, "IOMMUFD_INTERNAL"); + +#if IS_ENABLED(CONFIG_IRQ_MSI_IOMMU) +/** + * iommu_dma_prepare_msi() - Map the MSI page in the IOMMU domain + * @desc: MSI descriptor, will store the MSI page + * @msi_addr: MSI target address to be mapped + * + * The implementation of sw_msi() should take msi_addr and map it to + * an IOVA in the domain and call msi_desc_set_iommu_msi_iova() with the + * mapping information. + * + * Return: 0 on success or negative error code if the mapping failed. + */ +int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_addr) +{ + struct device *dev = msi_desc_to_dev(desc); + struct iommu_group *group = dev->iommu_group; + int ret = 0; + + if (!group) + return 0; + + mutex_lock(&group->mutex); + if (group->domain && group->domain->sw_msi) + ret = group->domain->sw_msi(group->domain, desc, msi_addr); + mutex_unlock(&group->mutex); + return ret; +} +#endif /* CONFIG_IRQ_MSI_IOMMU */ From patchwork Sat Feb 8 09:02:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolin Chen X-Patchwork-Id: 863501 Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2058.outbound.protection.outlook.com [40.107.212.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA6731AF0D3; Sat, 8 Feb 2025 09:03:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.212.58 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739005408; cv=fail; b=Mj4eC2GGFA/lXXFR6hgr7Jaj8G9+N2BTjO5ofwzFxSdrHLfNRExz9k0cSwP/rjmklamBFEmGgKF7ca+togXn6ocdRbdfQ1oUKNUdLzTvJDsfkEZQnR5ylQIIg1dABx5gV8s4vCESfiXx4BuoQa3qoolwTpZICe/FW8iCEsOxcA0= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739005408; c=relaxed/simple; bh=d6+jF2U2ZWbLpVMIPJF+clrCop6SF8Vc8K1g1togajo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Vhhzlku4V8V+75Cup2VptiTA5+BZFa4DswdJveppT2R9rS+R6/B2BxiW6BLX98HsmCxb0g/xW/dztKcYNZML/gtFOInECizmXwXygHwTfpBr+Bc6fs2VsQZp7cIJS0ftqOvtgZLMSRTDYkIlD6nulIz8ZFLDeUOw1FfkAaB85hQ= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=m3QosPRP; arc=fail smtp.client-ip=40.107.212.58 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="m3QosPRP" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=BGFThbxP2rZOaM4wyOIpiQy0TSxEpu6rJA9FWlaWNvaR05ewzPo2O5YqDgpnddGkarxF32j6aXIQ3L51yyo6+By+U6SYtwFJ4ZYaVYryVll7YXCpnWv6uRF23z2x9DNF7h/9iVzO4yqKwSXeyHj5k9OTuReMTwWXdWl6MmgAWV0JXJk/JFMQgJlKkGoUn3/sPNaSUOH8pz+HDDMXmP4UeYwS83gLkksVv8ePuaRXm+V2NhKbUZ+ruHN2cIeV3UW5BQ9U6ERGiCd8fpNeO82o54wzsDdSBVXnxJi5ub2SBwYeImAR3WFGM4T/WuP/ihGZVjvg0wzb5MMF+RmmOWX0ww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KuAVRcp0auQx3ZZIamlwFrjYBduj4Uc0fmVc7Uo0S8g=; b=WkziUpoLY6Na+cRumz8bKISK4c/WnM7vUPRlf4jsxUuZ0bcGMczXG1CG8Ux5j3pXS+B/HYC5NeuONT0QlNChIYNKh3UB7tJP9Ii0hY3HsUl2Fw5sofZIaA7DsG4cLEsIyxeicf4NIEBujbP1+5y+YleBRhvq2zmNk/7EvFZ9ZZLQcOxdM1PYMuE1mVbeLAw1dNUUbVfG3xKuz+HLtLAFD7Xn2z911FfFs4a2jBdFL+geY5nPsCFSuCRHUeA8SoNHNk964iV7cb7Hp24/Bg5vnNqrriJuP3tYu69KgBwavgvnv0cYgdt9Z5DjWwy5N7zmIJOmPNBlf8q/8JxUCw0RwA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=gmail.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KuAVRcp0auQx3ZZIamlwFrjYBduj4Uc0fmVc7Uo0S8g=; b=m3QosPRPGTtgoO2zVwQwUrKGmFakb5kLf+mQT+gseKiEifOdqE31CQ11o4IcA3zbxwZ6AweP1M1DFNCL2n6XhsSc7EQgRvS07adczSuc7lHtG+JUld/DS2ln8DX4ZTTWo0xl//CsXjafDufS7BW6qg4ywJk8k2glup4dHNUGTy2ducgZmoyNVFl71N0li6uYVfpFxiF8J3kb1LPJZ+FLK2TNTPVRXJVVTnOnbxUv9Aje7kBFcW43bDDXMGEbT8FM78Lvt67BHuxD+vioj08VOcQiciIHKVj4zwuYZfBjtqpGudimIHR8advTLwtNkjlQlBittWAnncmN/mJqOFD68w== Received: from DS7PR05CA0009.namprd05.prod.outlook.com (2603:10b6:5:3b9::14) by IA1PR12MB7496.namprd12.prod.outlook.com (2603:10b6:208:418::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8422.11; Sat, 8 Feb 2025 09:03:23 +0000 Received: from DS3PEPF000099D5.namprd04.prod.outlook.com (2603:10b6:5:3b9:cafe::93) by DS7PR05CA0009.outlook.office365.com (2603:10b6:5:3b9::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8398.24 via Frontend Transport; Sat, 8 Feb 2025 09:03:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DS3PEPF000099D5.mail.protection.outlook.com (10.167.17.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8398.14 via Frontend Transport; Sat, 8 Feb 2025 09:03:21 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sat, 8 Feb 2025 01:03:12 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sat, 8 Feb 2025 01:03:11 -0800 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Sat, 8 Feb 2025 01:03:10 -0800 From: Nicolin Chen To: , , , CC: , , , , , , , , , , , , , Subject: [PATCH v1 04/13] irqchip: Have CONFIG_IRQ_MSI_IOMMU be selected by the irqchips that need it Date: Sat, 8 Feb 2025 01:02:37 -0800 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF000099D5:EE_|IA1PR12MB7496:EE_ X-MS-Office365-Filtering-Correlation-Id: af5c9531-7f21-43c3-0f4c-08dd481f6fe7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|376014|7416014|36860700013|82310400026; X-Microsoft-Antispam-Message-Info: qyy3qg75/9Dld3pbdIomKWQUROeeAqWj1oMPRYf+8IHDjubVc46uLRfPo3kgPrshDI3jm3tpK+HaSzF41igPO6gwF0zctMfwneKMR7VxRrLe0/82mw8o1+rwoWyeGgktHSsAqn9mXqSLwnaR0xLSGyTnKauo2ZHZVIsvW8EleC1y/2MQk2/54n5NX8iN57QpBrq4cesj/Ty466ioWvr5E8VtJe6B8VaBY4M84a55PTzdEDas1+gtxPPKCPGmJecWLmWe1qq13sulAGLtbYQUV66mAn6vPc0epG14ooLdJaHphKmtVLZCAHXobfQ6wpU9fRxmkwZRsV62g/fChP0ryNzi3vcGlAdWfV2xCnJLwyRMeY8OiBpVTLxYfktjvtFDYYgDEOBP80XT1QOKhjMbXXJ7zgiY3RZV3NhAqXhhTm5CmgHLdpvaSjPnZZu5cA5S5lualGibdNjx1jhQqeKYmv2t8gaP4L3QcbCknaXBZpYBHHTgNxPa+MuIKk9Zg/b2u7ULQD2ZyQY7+np3B01rz8H0gpz6ZBJvsr0Rr0+nVlkSP5QXTtErJkl5V7bfPkh/J9OReI9bz8hSa70sgZVIugnwaMxm2sKeLXPUNcuCTeFL+BoFwb3H2vMGDB0AssS2ommrW7fFMvPb1ZYKBurrghPiyEEhjIlqdR8XhzQEvwskh8Gb6tcsOZSY4JbUkISvzZaNjlY97w4S5ZrOOWn9GuTEtx9JhPe1usahGhj/u4wwnDRPF6Ub5xUp33Fi9pBq548zG1aRcgH3C6pS8/FzzsIipVBkc1OpERv+kZh8K7mGhQCgVx88pt7ASveCSRoaPU/EJjI67DOujxadhbTDUbjesHh82ksEWgmQzcSSK9IBgRRjJLL06BAg64oFSHx1+GE8sNIRHUBbqUSIhaeB69YFuMfle0Eq6SZDZoGaJe6caDZVCJUn+6i1bUt568QbBvZDFN+T9Gdbx9IilOshCeB+bjXXzlPCy7QZfltb+zVzyT8uJL5xAowHqybZBhlc0GVcuJ/D6jNUGqd4aSI70dvyBmIvqOdFCWMzv5qzv3YDdbcCdctmIF/ycbvzfvor6UmmZt2J0V98sFmzkWR1qI6yE6r/a5gpn0h11D5cELIjMGabWXuKtzxdwSJW7IfzbMqRvB9bkb+v1xAA7suebAivab9Q4AsO+U5QqmxxBjG8t/or00Dku+ySxz6Rh9hhYQbRTnhOdisp3wzJKq0xU5/z5kU+IU2wyTZHUkgIOwVRN8c5h2tqWrlZkUlJjVbMH9J/jr8rmj5u7/LVJwUlCUiGuZZP7T04JLH/x/p3PknKUqVQ+Wej7Kse8vTQj9wQLmxRfjTLHAMbwpboXJLuA78hsGjzrU0M6Nr4kibvQ/mbqTGW/PNyeOasSw2PPHg92lM+SN886g7a4HogHMhOh1kft84Iq5ul7CXMH5I6QyCCzwA66le7OeT/fyD9Mqwil8p2ooR/SJMEFbOh/czQUQ== X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(1800799024)(376014)(7416014)(36860700013)(82310400026); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Feb 2025 09:03:21.4140 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: af5c9531-7f21-43c3-0f4c-08dd481f6fe7 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF000099D5.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB7496 From: Jason Gunthorpe Currently IRQ_MSI_IOMMU is selected if DMA_IOMMU is available to provide an implementation for iommu_dma_prepare/compose_msi_msg(). However it makes more sense for the irqchips that call prepare/compose to select it and that will trigger all the additional code and data to be compiled into the kernel. If IRQ_MSI_IOMMU is selected with no IOMMU side implementation then prepare/compose will be NOP stubs. If IRQ_MSI_IOMMU is not selected by an irqchip then the related code on the iommu side is compiled out. Signed-off-by: Jason Gunthorpe Signed-off-by: Nicolin Chen --- drivers/iommu/Kconfig | 1 - drivers/irqchip/Kconfig | 4 ++++ kernel/irq/Kconfig | 1 + 3 files changed, 5 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig index ec1b5e32b972..5124e7431fe3 100644 --- a/drivers/iommu/Kconfig +++ b/drivers/iommu/Kconfig @@ -154,7 +154,6 @@ config IOMMU_DMA select DMA_OPS_HELPERS select IOMMU_API select IOMMU_IOVA - select IRQ_MSI_IOMMU select NEED_SG_DMA_LENGTH select NEED_SG_DMA_FLAGS if SWIOTLB diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig index be063bfb50c4..84015decc3a4 100644 --- a/drivers/irqchip/Kconfig +++ b/drivers/irqchip/Kconfig @@ -28,6 +28,7 @@ config ARM_GIC_V2M select ARM_GIC select IRQ_MSI_LIB select PCI_MSI + select IRQ_MSI_IOMMU config GIC_NON_BANKED bool @@ -38,12 +39,14 @@ config ARM_GIC_V3 select PARTITION_PERCPU select GENERIC_IRQ_EFFECTIVE_AFF_MASK if SMP select HAVE_ARM_SMCCC_DISCOVERY + select IRQ_MSI_IOMMU config ARM_GIC_V3_ITS bool select GENERIC_MSI_IRQ select IRQ_MSI_LIB default ARM_GIC_V3 + select IRQ_MSI_IOMMU config ARM_GIC_V3_ITS_FSL_MC bool @@ -407,6 +410,7 @@ config LS_EXTIRQ config LS_SCFG_MSI def_bool y if SOC_LS1021A || ARCH_LAYERSCAPE + select IRQ_MSI_IOMMU depends on PCI_MSI config PARTITION_PERCPU diff --git a/kernel/irq/Kconfig b/kernel/irq/Kconfig index 5432418c0fea..9636aed20401 100644 --- a/kernel/irq/Kconfig +++ b/kernel/irq/Kconfig @@ -100,6 +100,7 @@ config GENERIC_MSI_IRQ bool select IRQ_DOMAIN_HIERARCHY +# irqchip drivers should select this if they call iommu_dma_prepare_msi() config IRQ_MSI_IOMMU bool From patchwork Sat Feb 8 09:02:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolin Chen X-Patchwork-Id: 863499 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2051.outbound.protection.outlook.com [40.107.220.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B4F521B4244; Sat, 8 Feb 2025 09:03:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.220.51 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739005412; cv=fail; b=S9kcelksJ2vvpBlD0lN08l9kaz7Bfl3X903XLCAb58fBal+B9jIbDDqqha3lbQhwfA/jmM67pY1p+a+0huw10JsAjNr6dwtykMdxL3h76orWo+asTGoL0B9Q3Mlc/LYB+cFzJ+V1ndst2IzgIJOTCVJhlt8LgSY9IiJV29gNe2k= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739005412; c=relaxed/simple; bh=cqnPUT+GZ/17/t7xPi8rJ50+GNVaYQnxkAf+bZNzgLA=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=gqPWv9RnWZHoKFepZpjN87W6hfqBruEiOmJt1C4l8LeN+4Nsz3K9/Vfj9iKmV10Dm73+vyUMnMJSr8/rw16GSDDkksqekxMgPQEFMPGgtDnOSMzytP+CURjmB7CDkPJbluIV4JBLf40NScavDsA3K4jIjxvkUS4JHbA64OvXNys= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Bep04BLb; arc=fail smtp.client-ip=40.107.220.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Bep04BLb" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kKjM0ML48Hou3qmrtShGDzIvlM2Z0kEnNNRiDdG6EfTRNBr+Hppfie21HOTD3g6TeZ10GX6pPgn2cavYEUqjgIYlV2sJQxo0VsuiSuipAsccFC/4DHsqvcwbRhklT9rjb6QkgL8b4SlhUujyCVONn2juUuq9HqbRkoQnLuiRBJ/FTeDn7xz2FxdTW5DxBXwL9vDBh2UCvHp9kwBZ0HWLzqjo0VqGTiQNHBuqtS77GlMgaSrfpziY+WH2jRdQQJlvzW4sZ6DPBh0TUhIdoVfc45UqeLtvscsRXdqY8lS5yG61fcv6aSDuofbtWULEzdlv4uhvNW6yDxsEUkEPM8eaNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wpvJ+vhxt8A3FUWSIdPivIO1epes2LJnk+IzWtZEkpg=; b=UBQjgxScRPliN4QTNRTdLj9he0+nyaBB7HFzwJ9STNNJSPjm1Utk0sxk8aVv36rI+kwxAkM1kEzyEwt0lMTpC53vSUbZ8q+4KBPkRC2VVgTjyKBXp1NmVgNlzXOUOxfryM+XyGnUAYNuovmd+BFoDRpu669lYmVCFvsG5dFQDQOyjMZrANIdgj7AmsOpWmaEQM1m3CQqyQeLzDJ4pfX6CC+Psk5q/f0zJQX+almjg+GBPJFOjaU1JhPFoH2aCS+hg0RonRlvjOTztHb93/KIxSvVGseMC72SzDpZklByAAvXM7skEHjgUGSrS+QVFGYd/qbBX6Li0u+jkA9+UxjCog== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=gmail.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wpvJ+vhxt8A3FUWSIdPivIO1epes2LJnk+IzWtZEkpg=; b=Bep04BLb3gWUpmEVQsL7mp2erZ+I1EDgVbcGyH4ZFipRtKZec7YgS8mtjmnKG8eDN72Bo6vxviQar1gPbv4a0rFtMBiO9WpZD10PCt8zBhKr1Oc/LD3QGQJMpxhIZsvTciYxCtT+mddgpdh/EMz2Yz0piUabd3lxLMTZo1XdYsdwm4CsSWZdEp7f9xdAKRLqzsMNimN61JKq+z3SCd6eyGZu84ahrXEQSNbH4Qk8DZTmHcJY1dwq3SDQOnh5YNScKtAiEVjwzctY/BnHXGa4uSzO7P8Gml76SQjSxw6xvAlwHHYpzGNYOUAinEIMJ9xKgfnBphSdSfTWYrhjd16xBQ== Received: from MN2PR18CA0010.namprd18.prod.outlook.com (2603:10b6:208:23c::15) by CH3PR12MB9282.namprd12.prod.outlook.com (2603:10b6:610:1cb::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8422.15; Sat, 8 Feb 2025 09:03:27 +0000 Received: from BN1PEPF0000468A.namprd05.prod.outlook.com (2603:10b6:208:23c:cafe::d1) by MN2PR18CA0010.outlook.office365.com (2603:10b6:208:23c::15) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8398.25 via Frontend Transport; Sat, 8 Feb 2025 09:03:26 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN1PEPF0000468A.mail.protection.outlook.com (10.167.243.135) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8398.14 via Frontend Transport; Sat, 8 Feb 2025 09:03:25 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sat, 8 Feb 2025 01:03:13 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sat, 8 Feb 2025 01:03:12 -0800 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Sat, 8 Feb 2025 01:03:12 -0800 From: Nicolin Chen To: , , , CC: , , , , , , , , , , , , , Subject: [PATCH v1 05/13] iommu: Turn fault_data to iommufd private pointer Date: Sat, 8 Feb 2025 01:02:38 -0800 Message-ID: X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF0000468A:EE_|CH3PR12MB9282:EE_ X-MS-Office365-Filtering-Correlation-Id: 8801e253-3d34-4638-70f3-08dd481f72a6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|82310400026|7416014|376014|1800799024|36860700013; X-Microsoft-Antispam-Message-Info: vlV+G0+8ONmWC4MZtVHlqtSlindinUYdZl8FS0Cb683K86YBQkvl0RNDpPI4OwsFg8VilQZzP2rEpJ/ByAO7YFjRlub2eAO/A1pQv3EURr+trBuEpYTK/6xcYz5R1Kefba9IR4auqSen9I31iduo+823XboSXngD9dy7UBlnUHoJPo9IxcEP9pkon07tgeRigp14Hymb63oIBeT4wSQqBvaWD/C7BeW4uOcFZmx3tEsOD6Myi9REsTvMwmC5Ip+afbSZiD0YhCj/j8PfNCe4viyVvZ1bHaWhVXfdAviscg+30j8ka/eaMHB69WUxHDjGWnxqVIgbkxwTfXqdE4GeU1wugva+4UQfQpI2ssRF4Z+RyibmEhWpIM+zQPVkrfOyEl56krqdn98btfZsa1OwzZgDuZ3ziu0OUiSyJ6qdHq+AyzdVicgeUFrO3FURi0QgQMKi/YSzItnUfOgiqtNsYkpDWTGs4jnMuV17KhoYy0D3qE/fzzCae6VIUMQcTvI21NwgaVq5v8CyzJkohtg/YpvI0pkFGRuNq1nrXaXfvQypYKZKRt3WPrw94TxqesrcLBx/INUS2D9sNTt9kxfSwSndpFrt30G5K15Cz5iYFDALI8W/ISuzQQUeWOG/2QlbyHy0LMHIGzoqhLi6hquajN8nOlcNnowouFwy2uZ8wa0ZKgav+cotm3WThdmRompEtjXFKIB1cBCjar0p2pUINElbgkrT7RJ+3ilDXv5Fl+JQ0NRlG4ZZ/aRj+2CJc446kfQIygggFQeME9vDPPDvibU9TzxbK+PjMYDwuJ2Y15wlg8cM61+fg6Sp4LGDWVWYTxNznIGpUTPffFjXctwul+zVhBYuyGrhdpi0lEpzqry33zNn1TRGcCdPY5bB/EbfaDmDyQ5fboXFWhD5qRqP2+CpZBNVJlbrFBcvDJA43/0BYT2wGHKJw3ltjQ7azHVsilXovzcwedAtyvwJ/hWgBcB5QzG26gYzvfFJeD5rY2GGyfm9fMCTVBBw0eUbzWS2UOhKGIxrRmc22bDXrzyOXKIVkhPnlBaEdriLCfBEaajaxOMzlRsQRRDcKGkS39GyjVNQpgGtaRGuY9s1ODgfpWn9sNcoQexHtXdwtWu2RLrdH0nSkz2DeQhtgZhRgHHxFh/77WTn68b8Rhhr3BA3tXoCbybzKXRnH65pVcJuv92KY11VbM/XP659GAWgJV8yTk+NVdvGnowUYpu9OJaiYW+B+omaHO9GTTA0fA0tEjAXSyso6UOQOCgygdofpyDzqIk48An34UvD/uN642EaEsFeyPFNu2jsMD9V2l0hqFuNKPNHdYfp4ADRCo5Ob8gK57onrYxESW8U6rILdV+/1DHXNMKV9ulBqAjLzjwW+g2y0XA1A3uewNaLGVzfRuVSuFqc15S9zPIcZpKlu/dzpsQdeZnPEt6xgJM9N8EnPMOMfnIRd3/1XlNTckI81St6UFDi8YJTXSz77OprkZNgXNi5ve9xtI6yoSqfMTLVDf8= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(82310400026)(7416014)(376014)(1800799024)(36860700013); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Feb 2025 09:03:25.9143 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 8801e253-3d34-4638-70f3-08dd481f72a6 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF0000468A.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9282 A "fault_data" was added exclusively for the iommufd_fault_iopf_handler() used by IOPF/PRI use cases, along with the attach_handle. Now, the iommufd version of sw_msi function will reuse the attach_handle and fault_data for a non-fault case. Rename "fault_data" to "iommufd_hwpt" so as not to confine it to a "fault" case. Move it into a union to be the iommufd private pointer. A following patch will move the iova_cookie to the union for dma-iommu too, after the iommufd_sw_msi implementation is added. Since we have two unions now, add some simple comments for readability. Signed-off-by: Nicolin Chen --- include/linux/iommu.h | 6 ++++-- drivers/iommu/iommufd/fault.c | 2 +- drivers/iommu/iommufd/hw_pagetable.c | 2 +- 3 files changed, 6 insertions(+), 4 deletions(-) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 761c5e186de9..e93d2e918599 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -224,8 +224,10 @@ struct iommu_domain { phys_addr_t msi_addr); #endif - void *fault_data; - union { + union { /* Pointer usable by owner of the domain */ + struct iommufd_hw_pagetable *iommufd_hwpt; /* iommufd */ + }; + union { /* Fault handler */ struct { iommu_fault_handler_t handler; void *handler_token; diff --git a/drivers/iommu/iommufd/fault.c b/drivers/iommu/iommufd/fault.c index 931a3fbe6e32..c48d72c9668c 100644 --- a/drivers/iommu/iommufd/fault.c +++ b/drivers/iommu/iommufd/fault.c @@ -329,7 +329,7 @@ int iommufd_fault_iopf_handler(struct iopf_group *group) struct iommufd_hw_pagetable *hwpt; struct iommufd_fault *fault; - hwpt = group->attach_handle->domain->fault_data; + hwpt = group->attach_handle->domain->iommufd_hwpt; fault = hwpt->fault; spin_lock(&fault->lock); diff --git a/drivers/iommu/iommufd/hw_pagetable.c b/drivers/iommu/iommufd/hw_pagetable.c index 598be26a14e2..2641d50f46cf 100644 --- a/drivers/iommu/iommufd/hw_pagetable.c +++ b/drivers/iommu/iommufd/hw_pagetable.c @@ -406,10 +406,10 @@ int iommufd_hwpt_alloc(struct iommufd_ucmd *ucmd) } hwpt->fault = fault; hwpt->domain->iopf_handler = iommufd_fault_iopf_handler; - hwpt->domain->fault_data = hwpt; refcount_inc(&fault->obj.users); iommufd_put_object(ucmd->ictx, &fault->obj); } + hwpt->domain->iommufd_hwpt = hwpt; cmd->out_hwpt_id = hwpt->obj.id; rc = iommufd_ucmd_respond(ucmd, sizeof(*cmd)); From patchwork Sat Feb 8 09:02:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolin Chen X-Patchwork-Id: 863498 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2051.outbound.protection.outlook.com [40.107.223.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 722AB1ACEBE; Sat, 8 Feb 2025 09:03:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.223.51 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739005420; cv=fail; b=Re27NooZNNGApd4ZUbVgqxEs6eRXvg1vQPKE4zQWalc8CwJyfzSB3mShElrzImfBwnm3rN5m4I0p1/XKra1LT2Y5pTubfhMUp4UuBFZiH/zXYI2jO3uvm8cI6nqnH/q2hB3No7qYPbv4dNjklY+YjZKRN9mceauA+iNlXj//xhw= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739005420; c=relaxed/simple; bh=8IgbYt7sjPJ6J73A3RwVoBtt7FS81878575Tj1HWNNQ=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=f+Gr5dWEK9igxuVSoD6mSOa3AbdqlzMY4Wo1PEgCv+lwd5f2Nywq9DXeVyqtU/a3WJsgoiacMYKUMD5BYW5vPNHHjzHFO/Hb7/6G91OhIY+aw+eRlFrkdOxEjJtoBMWM8RIbwdjecJtpLis50ndJsbgMhcQs8ZcC1fXnsH1WFO4= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=MemCyHvM; arc=fail smtp.client-ip=40.107.223.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="MemCyHvM" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=qdAnc1rvUbe7RN5szDbVsgCQMDhUPnnn8DjmEjIjkWDFN6Ba0b5awBHKEB5RJScZbFSJ13P3J4fWkI27zNL4Y2U5HxCH8bkMdDg2mXiy/YoyNrOdhOzZ5WdP5cb8q/vz+LB+HDg0KtEuFAeB9SCE+bQ//lvG2xBxW+/b1rrOtrWvmx9i9emyU0YXq86D7aADLUZ0sjVWjXnTYvrWmFNa/qHFbt/Khl9WKFIiippm6EyLTiC4W9YuEj5h0i8A1ZwP+o7MAR4iM85oYm8WdNLRl9g1w1e5WJlg1GAxs8vGgddseaPb3Ykmbk+qiwS8QoNTX7fRzRlZO0l4dntGpz9rPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+iqipAWJNsOzidSS57UFbifnGAokb7H+HFB6FjfIMx0=; b=HjPZCaJy5Mtbg/bXtJwTFPumXjCOb5frXTfyHIx+c4ZqPf4qV320f6hy0K/cumjRtdbhzIVyuIvfbYNdCwV+MakwHvE+ZciTgU6WRCKIzJGBnU6FJFHuXd4ipReO3zqFyvDLnkBCe+2R0RMfqGL9tGiwFmnAMt92fZi3dtxgL2mHKadCYrPUEZ1C04MjqUcABx8e7sAWoCarylLBY++PmOOAs9dV/ZAICnmgBTPUbGKAfGKcNwYKrrHdp52igTd0p0Nmi4pNRMvsINbWyGBdQhuOu/t+hCHcAYz5q+FtsXX1c4vfSGVUn/c3VoMPdIMbVvLui7NRkdtuRU3hLa7eJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=gmail.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+iqipAWJNsOzidSS57UFbifnGAokb7H+HFB6FjfIMx0=; b=MemCyHvMb+wWpIV7tTBn/7Cyukw1ABlFf3ISKrvgtuT/Ed2mBVwrBEgvsyRvXxzf+k/gXiZ5a08r6B8X1Y6QB4lTSuleQtzEb9AVyH6CgRuwrYSnZp2b0BdowBmPz58ZNaiWs0+4K1DYXWL4vpO0V+FjEjzM63B5siWHTX6/zRgYhHLsWURVg8EIMwynRVyA4yg8AWGJ5QXH85oz/fW8/XYy2j2Iamih4p3TtrGWIICAQ+13WQLm+Sc/6KUFHU7bXe24v5+LKt7XQ7L6ELA1/70qMcmaJVEsigtpbu8q8pDDY5UjmhLDbGZd+841DAf+CGGd5eXCfm/+aco4iHrNOg== Received: from DS7PR05CA0009.namprd05.prod.outlook.com (2603:10b6:5:3b9::14) by CH3PR12MB9252.namprd12.prod.outlook.com (2603:10b6:610:1ba::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8422.14; Sat, 8 Feb 2025 09:03:32 +0000 Received: from DS3PEPF000099D5.namprd04.prod.outlook.com (2603:10b6:5:3b9:cafe::8d) by DS7PR05CA0009.outlook.office365.com (2603:10b6:5:3b9::14) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8398.24 via Frontend Transport; Sat, 8 Feb 2025 09:03:32 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DS3PEPF000099D5.mail.protection.outlook.com (10.167.17.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8398.14 via Frontend Transport; Sat, 8 Feb 2025 09:03:31 +0000 Received: from rnnvmail204.nvidia.com (10.129.68.6) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sat, 8 Feb 2025 01:03:19 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail204.nvidia.com (10.129.68.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sat, 8 Feb 2025 01:03:18 -0800 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Sat, 8 Feb 2025 01:03:17 -0800 From: Nicolin Chen To: , , , CC: , , , , , , , , , , , , , Subject: [PATCH v1 10/13] iommufd: Add IOMMU_OPTION_SW_MSI_START/SIZE ioctls Date: Sat, 8 Feb 2025 01:02:43 -0800 Message-ID: <525b9ffa94dc5842f98bef8cbcbf426a2a8292be.1739005085.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF000099D5:EE_|CH3PR12MB9252:EE_ X-MS-Office365-Filtering-Correlation-Id: 35d90a11-432a-4a8f-26ab-08dd481f7635 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|82310400026|7416014|36860700013|376014; X-Microsoft-Antispam-Message-Info: WiwMPYcIuh40MRuOKu3GaZ49WT2CE7+uKgm3XDXWj8+q8szLkqHamcUd1Dbti0vTo+QHnS4QNAYZfTkf36DPGyOcxRAdFkT5tClhtp3cijZf6npeme+hsrwPxx736mCgyFG+hOLvYP2iKsp/BXnkF1Utb1fzabhEXCOdsA2DkBST/usASzaaoqnX+/1hO42bCs5QoY7DVK23RUvqEa/1ZpTihgj55AW4fsQVSdFayAroKwa1hWFyrWP/rYtRGu04iVM7f6KISsiaQMunRiA7klqS2rgC2lwEG6WvZkL3wxxwqtvFGfyqdR3Pt+MoEauOf0n+dTJX2HHosbyxaJWda+SWBQeE475QxxcUG3pL82Sp0TxKxpjIX9l9DC44o6QTnEydQuClMN9ODMd4YKa3WShKoo790Ionq3bEwif7NLHX/ERmQAmWHkC+5nJp6nrOwZnnps4ocRUAt21Ux5HXs0IYCCfQhpHFGZPd8Gby0Os8A2G7nfDC4mAR1jDID4+4wgm64t9wmboPceqbmof7QnPfpJswjbtVGU3VVLDvPyhuG4u0GpqKAngnUeiRy4+DlxNnZvZExmHkhmCe+ECuX+fJ2ZZv3m5hiQxNufq3/DG5lBcMib6qrZNHhHQdab0vwkG7uH4OTyujfHow92CtQuu0fDnVol7ZVrQTdPDsD/tSGlRMVWVR0WQ3Mw++UczOxxzE3XnZzeAFsf8IMxYezTPmyHyCe43XZDgKIWGt8hGSGDTsJhnXdxSLUjc+20oyRg8f8gCf9i/LyOL1jsg8MeU0ND8umVyKG4hRlDh2xcM+vD2Ca8LBYxZbXOVgKtG+FWgj99yOs5nsIPG3FTwEpVsTtXXtpCfyZsct7pOfJaLxMB9BukbeitkML7cy8dzatDyqeVoUwxtOgOLZh5E1Yz3ovvm++4XmMNmbec2CA4sliW5G01OAVwGNRdrZfAd5UKUjm5keJxgmwugt+XjbKrVMpkKTdAD/K0FKqw+GzAQO9mg19/9+YndfMzcVTbx8cp0Dft0pvBjxb4Y++wsJGcExHzoLxjUhoi57V23+hUH+2C/Su+d28ApwSn2UVMrsY10+IbHWt4hcBaXM6F3ZxIjjk9HMtl6NBkzWXykhObWtccO0YwUqy5WKVh8HCg2ygIO2OD22NvWkB+xLLyoHZXWC5O1xgY5S1fUrFrECjymepYEGplL0LQmMKorR+5REnLhYxqNMec4CkO42OydB3RX+Pz74u48JKdKWqWl7waBWTE7pKuxz16s/hb37afkLLLSy9sPi9qPWa+RkIJvKvwWYbOv+8vM1QjHCK+LZsYqWFKvwbpmRfmAmXbIhTku9OZakrXLVfFS1PudSUKdY88mJ5mckKeL5AFrRyRrY0iiMraK0AWF2MhiY5QxdR3BX9iHBkfCpmjP1dUtkNQm0qHDSwm91EvAa0tMfdIuPug8nQqyUDJjT1Dpp7hlH0wpsDPbzjChlCM+8JDFFFoDITgU4Y92mYy8Pj2l/XUGftVI= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(1800799024)(82310400026)(7416014)(36860700013)(376014); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Feb 2025 09:03:31.9453 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 35d90a11-432a-4a8f-26ab-08dd481f7635 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF000099D5.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9252 For systems that require MSI pages to be mapped into the IOMMU translation the IOMMU driver provides an IOMMU_RESV_SW_MSI range, which is the default recommended IOVA window to place these mappings. However, there is nothing special about this address. And to support the RMR trick in VMM for nested translation, the VMM needs to know what sw_msi window the kernel is using. As there is no particular reason to force VMM to adopt the kernel default, provide a simple IOMMU_OPTION_SW_MSI_START/SIZE ioctl that the VMM can use to directly specify its desired sw_msi window, which replaces and disables the default IOMMU_RESV_SW_MSI from the driver, to avoid having to build an API to discover the default IOMMU_RESV_SW_MSI. Since iommufd now has its own sw_msi function, this is easy to implement. Keep these two options per iommufd_device, so each device can set its own desired MSI window. VMM must set the values before attaching the device to any HWPT/IOAS to have an effect. Suggested-by: Jason Gunthorpe Signed-off-by: Nicolin Chen --- drivers/iommu/iommufd/iommufd_private.h | 2 + include/uapi/linux/iommufd.h | 20 ++++- drivers/iommu/iommufd/io_pagetable.c | 15 +++- drivers/iommu/iommufd/ioas.c | 97 +++++++++++++++++++++++++ drivers/iommu/iommufd/main.c | 4 + 5 files changed, 134 insertions(+), 4 deletions(-) diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h index 7a9cc6e61152..2d1aae7c8610 100644 --- a/drivers/iommu/iommufd/iommufd_private.h +++ b/drivers/iommu/iommufd/iommufd_private.h @@ -279,6 +279,7 @@ int iommufd_ioas_change_process(struct iommufd_ucmd *ucmd); int iommufd_ioas_copy(struct iommufd_ucmd *ucmd); int iommufd_ioas_unmap(struct iommufd_ucmd *ucmd); int iommufd_ioas_option(struct iommufd_ucmd *ucmd); +int iommufd_option_sw_msi(struct iommufd_ucmd *ucmd); int iommufd_option_rlimit_mode(struct iommu_option *cmd, struct iommufd_ctx *ictx); @@ -423,6 +424,7 @@ struct iommufd_device { struct mutex iopf_lock; unsigned int iopf_enabled; phys_addr_t sw_msi_start; + size_t sw_msi_size; }; static inline struct iommufd_device * diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h index 78747b24bd0f..310256bc3dbf 100644 --- a/include/uapi/linux/iommufd.h +++ b/include/uapi/linux/iommufd.h @@ -294,7 +294,9 @@ struct iommu_ioas_unmap { /** * enum iommufd_option - ioctl(IOMMU_OPTION_RLIMIT_MODE) and - * ioctl(IOMMU_OPTION_HUGE_PAGES) + * ioctl(IOMMU_OPTION_HUGE_PAGES) and + * ioctl(IOMMU_OPTION_SW_MSI_START) and + * ioctl(IOMMU_OPTION_SW_MSI_SIZE) * @IOMMU_OPTION_RLIMIT_MODE: * Change how RLIMIT_MEMLOCK accounting works. The caller must have privilege * to invoke this. Value 0 (default) is user based accounting, 1 uses process @@ -304,10 +306,26 @@ struct iommu_ioas_unmap { * iommu mappings. Value 0 disables combining, everything is mapped to * PAGE_SIZE. This can be useful for benchmarking. This is a per-IOAS * option, the object_id must be the IOAS ID. + * @IOMMU_OPTION_SW_MSI_START: + * Change the base address of the IOMMU mapping region for MSI doorbell(s). + * This option being unset or @IOMMU_OPTION_SW_MSI_SIZE being value 0 tells + * the kernel to pick its default MSI doorbell window, ignoring these two + * options. To set this option, userspace must do before attaching a device + * to an IOAS/HWPT. Otherwise, kernel will return error (-EBUSY). An address + * must be 1MB aligned. This option is per-device, the object_id must be the + * device ID. + * @IOMMU_OPTION_SW_MSI_SIZE: + * Change the size (in MB) of the IOMMU mapping region for MSI doorbell(s). + * The minimum value is 1 MB. A value 0 (default) tells the kernel to ignore + * the base address value set to @IOMMU_OPTION_SW_MSI_START, and to pick its + * default MSI doorbell window. Same requirements are applied to this option + * too, so check @IOMMU_OPTION_SW_MSI_START for details. */ enum iommufd_option { IOMMU_OPTION_RLIMIT_MODE = 0, IOMMU_OPTION_HUGE_PAGES = 1, + IOMMU_OPTION_SW_MSI_START = 2, + IOMMU_OPTION_SW_MSI_SIZE = 3, }; /** diff --git a/drivers/iommu/iommufd/io_pagetable.c b/drivers/iommu/iommufd/io_pagetable.c index 441da0314a54..6e6dcc480922 100644 --- a/drivers/iommu/iommufd/io_pagetable.c +++ b/drivers/iommu/iommufd/io_pagetable.c @@ -1441,18 +1441,27 @@ int iopt_table_enforce_dev_resv_regions(struct io_pagetable *iopt, iommu_get_resv_regions(dev, &resv_regions); list_for_each_entry(resv, &resv_regions, list) { + unsigned long start = PHYS_ADDR_MAX, last = 0; + if (resv->type == IOMMU_RESV_DIRECT_RELAXABLE) continue; if (sw_msi_start && resv->type == IOMMU_RESV_MSI) num_hw_msi++; if (sw_msi_start && resv->type == IOMMU_RESV_SW_MSI) { - *sw_msi_start = resv->start; + if (idev->sw_msi_size) { + start = *sw_msi_start; + last = idev->sw_msi_size - 1 + start; + } num_sw_msi++; } - rc = iopt_reserve_iova(iopt, resv->start, - resv->length - 1 + resv->start, dev); + if (start == PHYS_ADDR_MAX) { + start = resv->start; + last = resv->length - 1 + start; + } + + rc = iopt_reserve_iova(iopt, start, last, dev); if (rc) goto out_reserved; } diff --git a/drivers/iommu/iommufd/ioas.c b/drivers/iommu/iommufd/ioas.c index 1542c5fd10a8..1fc93bc70bf4 100644 --- a/drivers/iommu/iommufd/ioas.c +++ b/drivers/iommu/iommufd/ioas.c @@ -620,6 +620,103 @@ int iommufd_option_rlimit_mode(struct iommu_option *cmd, return -EOPNOTSUPP; } +static inline int iommufd_option_sw_msi_test(struct iommufd_device *idev, + phys_addr_t start, size_t size) +{ + const phys_addr_t alignment = SZ_1M - 1; + struct iommu_resv_region *resv; + phys_addr_t resv_last, last; + LIST_HEAD(resv_regions); + int rc = 0; + + /* Alignment Test */ + if (start & alignment) + return -EINVAL; + + /* Overlap Test */ + if (!size) + size = SZ_1M; + last = size - 1 + start; + /* FIXME: drivers allocate memory but there is no failure propogated */ + iommu_get_resv_regions(idev->dev, &resv_regions); + list_for_each_entry(resv, &resv_regions, list) { + if (resv->type == IOMMU_RESV_SW_MSI || /* iommufd will bypass */ + resv->type == IOMMU_RESV_DIRECT_RELAXABLE) + continue; + resv_last = resv->length - 1 + resv->start; + if (resv->start <= last && resv_last >= start) { + rc = -EADDRINUSE; + break; + } + } + iommu_put_resv_regions(idev->dev, &resv_regions); + return rc; +} + +int iommufd_option_sw_msi(struct iommufd_ucmd *ucmd) +{ + struct iommu_option *cmd = ucmd->cmd; + struct iommufd_device *idev; + int rc = 0; + + idev = iommufd_get_device(ucmd, cmd->object_id); + if (IS_ERR(idev)) + return PTR_ERR(idev); + + mutex_lock(&idev->igroup->lock); + /* Device cannot enforce the sw_msi window if already attached */ + if (idev->igroup->hwpt) { + rc = -EBUSY; + goto out_unlock; + } + + if (cmd->op == IOMMU_OPTION_OP_GET) { + switch (cmd->option_id) { + case IOMMU_OPTION_SW_MSI_START: + cmd->val64 = (u64)idev->sw_msi_start; + break; + case IOMMU_OPTION_SW_MSI_SIZE: + cmd->val64 = (u64)idev->sw_msi_size / SZ_1M; + break; + default: + rc = -EOPNOTSUPP; + break; + } + } + if (cmd->op == IOMMU_OPTION_OP_SET) { + phys_addr_t start = idev->sw_msi_start; + size_t size = idev->sw_msi_size; + + switch (cmd->option_id) { + case IOMMU_OPTION_SW_MSI_START: + start = (phys_addr_t)cmd->val64; + rc = iommufd_option_sw_msi_test(idev, start, size); + if (rc) + break; + idev->sw_msi_start = start; + break; + case IOMMU_OPTION_SW_MSI_SIZE: + size = (size_t)cmd->val64 * SZ_1M; + if (size) { + rc = iommufd_option_sw_msi_test(idev, start, + size); + if (rc) + break; + } + idev->sw_msi_size = size; + break; + default: + rc = -EOPNOTSUPP; + break; + } + } + +out_unlock: + mutex_unlock(&idev->igroup->lock); + iommufd_put_object(ucmd->ictx, &idev->obj); + return rc; +} + static int iommufd_ioas_option_huge_pages(struct iommu_option *cmd, struct iommufd_ioas *ioas) { diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c index b6fa9fd11bc1..f92fb03ca3c1 100644 --- a/drivers/iommu/iommufd/main.c +++ b/drivers/iommu/iommufd/main.c @@ -287,6 +287,10 @@ static int iommufd_option(struct iommufd_ucmd *ucmd) case IOMMU_OPTION_RLIMIT_MODE: rc = iommufd_option_rlimit_mode(cmd, ucmd->ictx); break; + case IOMMU_OPTION_SW_MSI_START: + case IOMMU_OPTION_SW_MSI_SIZE: + rc = iommufd_option_sw_msi(ucmd); + break; case IOMMU_OPTION_HUGE_PAGES: rc = iommufd_ioas_option(ucmd); break; From patchwork Sat Feb 8 09:02:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolin Chen X-Patchwork-Id: 863497 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2055.outbound.protection.outlook.com [40.107.244.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CED851AE00E; Sat, 8 Feb 2025 09:03:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.55 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739005422; cv=fail; b=knGRGJhYsk35bydVDYhiY873Nqs8Ht4aZ4Ra034yDAiyqiYkI7trWd2S3uJCJ18O67PTdsSfFbi1VDYcG84inkYVR439NZb7kYEo5ufHkEO15fsJcuWZdTet2KG2nkwaAvz8psf8ZNvsp/9tkYrT8rpM7KDCbeHBP+fxXdskdIs= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739005422; c=relaxed/simple; bh=1+eOWesKVuvqeNkiiUg21K+tQI+6q1exNdWqThNj8uE=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=dAtVpyCbzEiL64Qe5DR99uNYNO807819mi/oB5j30uY/QV1CC0l7id1/TANqsZnHdx8kWzdmtBu+xOnMhT9f4j6SXSQDZAAFghnOHnU382Iar344VELwqiuu27ZraOkBzqwMSYbaxTUvbTB74J1+UJ0hb/mpPDOmf/HmkTaxT9Q= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=UTsI5KZq; arc=fail smtp.client-ip=40.107.244.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="UTsI5KZq" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=k5LU1VzELO7u7B/QuDGoS3VNfa1Do9c6AgPkeLhgHBP9eC0dWuOrkCx4aKI5TqmybAFjw6c+WQDqcbuhgWmt+cEL/LMTojSbKth+HCWxqDDC4HfFUzxqB/xWYx1d4H3Vrpk6bPqdNBfUwddGLlz7LZHUMLqmFua1p7O5RX5MzWT2ovq51eUaMxQ2t1xR+P7e1E5foWy3dpgYymDQMJ+rUKK464KU845WFgJZvOJ6Ay8d/kF5OnYXM1rMvgUtofPe/6JvpZtnnkt9o2nyx+mzERlXb1sUxkMnru4g993S4puB8+H4hEDCkGqtUEnq2KAoSo1LUIYBOW8kBdQ9LAFw4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lHRb2/rDTR1ziVcb52A/gFWpDhGXPSIt+KjWcDb0ugs=; b=BQ7P0YvXu///4qL1qT67ClrwLsE9S5MgcOrZpmcWS6COSuHXOVkFoGnUhKXv8jFGt97cIZJHgFCgWAdtBgwvupcU/7Dzp/ZIs/hvEd9wawp7uY1FNR4mSq+XTiBWBYQrxqM98VT+w+WlG+T9DpWno1uwexPOvLayc/qqoN8p81m7pPgfLDRbnY1yP7HxAhpkJiItEkkNmPaMesk4OZ9g8rKrrNe7w6bl9qehXtmWSo/OxU2esJnlSsdPH0+P1sLJQMpImQq0q4mg7eJ7wX8h4fBMN0WLFk2GQv5+Igo6XSCMRAZBntHqCaPMvJnofEOeu2dovsO7NS8Ejpfq4h/MWg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=gmail.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lHRb2/rDTR1ziVcb52A/gFWpDhGXPSIt+KjWcDb0ugs=; b=UTsI5KZq0uXtY9Iw57PYn4NCaTExlUo5a0qfbTr203RYjcw9o+WyaA0Qn1LYv2TFKnewFV5EoY21TV/5fXHNJZ8OSE/RjPJYkALpdPdz/WVzOR6T1XN5MN3qCRkfiiissB+mmziZsLNrXNVoxw7DNzIQhl8mv4u9NQ9aQGKhAAd7Cc31tjq2i9sUJeHywb6XQezXhV57idOA5Zcw3h3UXLNVdsAD70zsnBFHOFZxBTuFO81rBuK0cmtwmnzPxt9NNjsA5fE6ZqlR+kgJ+ZHHZs8v3KAPdTD4QZ1sbXcdE24NSGWSeNElUBKGqCUj4okL807cxiU8hEVehngeVNM4rg== Received: from BL0PR02CA0111.namprd02.prod.outlook.com (2603:10b6:208:35::16) by DM4PR12MB5913.namprd12.prod.outlook.com (2603:10b6:8:66::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8422.10; Sat, 8 Feb 2025 09:03:37 +0000 Received: from BN1PEPF00004689.namprd05.prod.outlook.com (2603:10b6:208:35:cafe::fa) by BL0PR02CA0111.outlook.office365.com (2603:10b6:208:35::16) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8356.22 via Frontend Transport; Sat, 8 Feb 2025 09:03:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.161) by BN1PEPF00004689.mail.protection.outlook.com (10.167.243.134) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8398.14 via Frontend Transport; Sat, 8 Feb 2025 09:03:36 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sat, 8 Feb 2025 01:03:21 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sat, 8 Feb 2025 01:03:20 -0800 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Sat, 8 Feb 2025 01:03:20 -0800 From: Nicolin Chen To: , , , CC: , , , , , , , , , , , , , Subject: [PATCH v1 12/13] iommufd/selftest: Add a testing reserved region Date: Sat, 8 Feb 2025 01:02:45 -0800 Message-ID: <4882654aa73b39756393b12d7455626a3837b52c.1739005085.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN1PEPF00004689:EE_|DM4PR12MB5913:EE_ X-MS-Office365-Filtering-Correlation-Id: 9a55aee9-389b-4649-d597-08dd481f793d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|7416014|36860700013|82310400026|1800799024; X-Microsoft-Antispam-Message-Info: sg0WPKlvTh0YRIyQvhsq/Hfb230ckqPxQ+xToEHEhyc1QB7DtRepirNhCI8TWE5ZfPabgC/ijup6/laQhWjpbshJG5oMoOUncTCFA37qh1XZFcCXyLXsrRhsgUT0D/oQ1R/L5epiB/9xPMCY5ABOrzqFcg3ZGp8tnRCxleGW14Y+LicTZVzLuN3X2aIQ/HvqUOVfficPk9d4eH90BQzs9cOTPsUP+lO4t14MQIUb1+o50Lz6G2WVWq+7iW2fyDkm+CjuwOpAk3clN5Dvpu+HCXA8hRy6Ti3gjnCjq5XoS8i0OgahBLF8gEqIDRmQDtTmwG6MUKyHrdvfgh90qiFc2HFIfnQ39XHPMbuXkrbeg3fxrJtQPOFo8M2Pk9a272rwy8SnSNqlMpijBAz86PDmqKw/f7j5uvCJXbZiQjJslIPXQVCOzSNSbXnYh79XhycwulWE13xMKIaMlZZ6fhPFvxP4LcyUnHQbqdT3jZmu7aoisj4QiZxNre2iJk2RKIJqMadVrkrZvx6H2gjKpiPQfemPc5Y3G/hUkl80uYs32QL8Sv7pbttohCpk7zY2t9/btlO7PqLmhAHwN+xaNZT/ORDtU58uIPbF7DwnUq2gLNyTuEFpHP+2mqZU+VNf4CDpaCwnvXhxoq5klK112U+6gsjeVdZl7hnnS9/FZt7NQr40prHI93QbutJwkCRulxgeTP3ppXjudNS7XGUdAz1zEdlUyMs6Dscpq3ZUXawjVSgofTzSONUIFcfrO6wnG8gb3X2/fyjUepsPVC/HvEzL9nRnv4EedB2OMHn183WcCI8PFvZWJZajvEFHfpkgwGDF71tUZcayyz6epLfpwJya9ud7JdJaHa4D4rHr2nM/Ldh7ZlIxJ4x5P102vlmTxOS6KsEtthBeU2w6JBwktiTFatt31ObrBx89zy0+ZrEbF3QmPFvFTEMS7lhW9qfTxVZpqJ2pQ3UHnqKFXPTh0NGD809d+i8dLt1J3RQLocAKGtdhGCJhRM1S80H2yRlDFpkBS+0132Bq0bgD2AXWAciqIiw8AaeEVYNsnGTXFYizdm4HOHcSUb6djXNBIIJNzPpP0k6uoUfl5G3ZPuWl3NZpFc5SOX7H0/uWSo5rhxjGFhO0EkAkyE7eufiYWe33gnHGaSUqQBN/ZiPA+DP8Fh7CkdsXHRLxd97Orjd+PcxXvB39b52PRbkWUVVDDmQ2PA/uGWZVIQQ8hUxJaU6SfXjjLLQRqt8r54lqj1IUAh/soYUritUYWMF9zLDRKvUh6YMdeDJNMeWWLRA6rHM9/4dvrT2HQ5GV0F/nxdZhOy+Fss1emgs5Y999VoPeaNQvzQotJWoDre9/mZ89HH3eSjxIip+gJHvl2Fw0spr6goOZPaM4vZNWdkjXV8OCQZEvrZyFYk7V6pIWK6HFPsKLT03uDfIKGga0T5mq6p6k0lB7FgvPh2B41EguqrpFsF4TkWfAexUwblG13NwooK3hW55w7S9Ugy1pr+aPMqNOADiEwlU= X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230040)(376014)(7416014)(36860700013)(82310400026)(1800799024); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Feb 2025 09:03:36.9728 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9a55aee9-389b-4649-d597-08dd481f793d X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BN1PEPF00004689.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5913 The new IOMMU_OPTION_SW_MSI_START/SIZE must not overlap with any existing device reserved region, so add a testing region [0x80000000, 0x8fffffff], on top of the normal IOVA aperture for selftest program to run an overlap test. Signed-off-by: Nicolin Chen --- drivers/iommu/iommufd/iommufd_test.h | 3 +++ drivers/iommu/iommufd/selftest.c | 19 ++++++++++++++++++- 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/iommufd/iommufd_test.h b/drivers/iommu/iommufd/iommufd_test.h index 02be242f8f34..53e2e30570fc 100644 --- a/drivers/iommu/iommufd/iommufd_test.h +++ b/drivers/iommu/iommufd/iommufd_test.h @@ -213,4 +213,7 @@ struct iommu_viommu_invalidate_selftest { __u32 cache_id; }; +#define IOMMU_TEST_RESV_BASE 0x80000000UL +#define IOMMU_TEST_RESV_LENGTH 0x10000000UL + #endif diff --git a/drivers/iommu/iommufd/selftest.c b/drivers/iommu/iommufd/selftest.c index 37a5cb89e27c..f4ac443d73d6 100644 --- a/drivers/iommu/iommufd/selftest.c +++ b/drivers/iommu/iommufd/selftest.c @@ -13,6 +13,7 @@ #include #include +#include "../dma-iommu.h" #include "../iommu-priv.h" #include "io_pagetable.h" #include "iommufd_private.h" @@ -379,7 +380,8 @@ mock_domain_alloc_paging_flags(struct device *dev, u32 flags, if (!mock) return ERR_PTR(-ENOMEM); mock->domain.geometry.aperture_start = MOCK_APERTURE_START; - mock->domain.geometry.aperture_end = MOCK_APERTURE_LAST; + mock->domain.geometry.aperture_end = + MOCK_APERTURE_LAST + IOMMU_TEST_RESV_LENGTH; mock->domain.pgsize_bitmap = MOCK_IO_PAGE_SIZE; if (dev && mdev->flags & MOCK_FLAGS_DEVICE_HUGE_IOVA) mock->domain.pgsize_bitmap |= MOCK_HUGE_PAGE_SIZE; @@ -567,6 +569,20 @@ static int mock_dev_disable_feat(struct device *dev, enum iommu_dev_features fea return 0; } +static void mock_dev_get_resv_regions(struct device *dev, + struct list_head *head) +{ + const int prot = IOMMU_WRITE | IOMMU_NOEXEC | IOMMU_MMIO; + struct iommu_resv_region *region; + + region = iommu_alloc_resv_region(IOMMU_TEST_RESV_BASE, + IOMMU_TEST_RESV_LENGTH, prot, + IOMMU_RESV_RESERVED, GFP_KERNEL); + if (!region) + return; + list_add_tail(®ion->list, head); +} + static void mock_viommu_destroy(struct iommufd_viommu *viommu) { struct mock_iommu_device *mock_iommu = container_of( @@ -711,6 +727,7 @@ static const struct iommu_ops mock_ops = { .page_response = mock_domain_page_response, .dev_enable_feat = mock_dev_enable_feat, .dev_disable_feat = mock_dev_disable_feat, + .get_resv_regions = mock_dev_get_resv_regions, .user_pasid_table = true, .viommu_alloc = mock_viommu_alloc, .default_domain_ops = From patchwork Sat Feb 8 09:02:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nicolin Chen X-Patchwork-Id: 863496 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2058.outbound.protection.outlook.com [40.107.220.58]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE4671AF0B6; Sat, 8 Feb 2025 09:03:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.220.58 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739005425; cv=fail; b=uEkToUx5C4UmK4Mls47+3CSSBe76tanvlXf74YZTRqVmy6keRuoYILW015mBT9OCl48c0AH8O/ubJAl1UzGgUhEFfgrTqD1FbaufgsydbQMkZ8F4nn3Vcsi5iPV6TZJ/AwaFvbrJJwUSy3RHUjgatAhVbOY3GAPUMEQeVxok2So= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1739005425; c=relaxed/simple; bh=tfK4keXP6Utitp47u6p0wIc2KYrP4j4XoE0qhPeKxgw=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=rbjE+4WRRs8IzO4Ycuw5R8Cu0R5sdwyxdMNLZEsdyX08djAENYMUwq9rrxG3/rIgs4icjW9TcZXFcvldMnNIZ/BqAl9SaysgFt1Isyr1oQZSN78GIuBm5u0SXfo+dFOR9eePjtIP7HLZfPSWMoc8lJ833U7yD/vguOutoYi0FHo= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=YB79Cekl; arc=fail smtp.client-ip=40.107.220.58 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="YB79Cekl" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=RSARF7Rv+/eyjOOsFYRcq12E89SeHwDJQM0xYfXpuYkEIPfo3b/fjIrfsvfqjCUhq8ZJpdQbHArnMWCjh7fxcEVlUgfIL2XLGOCL2GRJi3wlII/t0EfT3BmpN37QKtmt74kVTDd8EvgOsIwt9Rd6BUjcwnGTVAqsFtk+PHeSWN12UOxYFRVbRFIG1eKCnwhSLoZU+jWCPno/OH5D8BJvA592yOKTJvuhPOMb0c7oz6XFrrBSCthWq3W11UCl68W6AS5x5oZAKjdGtr7+VjJvZxkOX8rJiYpNquA/bFn8WSGMbnV0oJatZepKmtnTM3OIKwJy/PyjwPK2XQU7qtpcGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=L1wQ1OlxyUNC6t/+EFekUqCIAmqr4wF1LuLBv5UCmok=; b=XeSpmntnH4zAQG7IwhZGEGRrQvyP5Jwx2OdDbmem2Q7HdZRpyhCYAVYzBUCiiD54E5xF9OeJNMUeZ4D0BOoieApo3ywmuvt5HqncQSX9mOXrl3AX5i/OdWs5PoQLWc1MoCXOMVOGzp2EkWzzI6wYSLo9h2NMWSidKB/FRjGN4RwOwLmD4o/OcPUCWxJL1/7+mLUsjbSaXMy8ct9dpoSQdTP2nwI3XsPu8Iki4feMCUzYK4SC2SKi73dSpzyaJ9YmddKZdE0J37KBVwJUoOf3u4NH2CLP3odNe/ehmRwwb2dSZWAAaJsXMTny6tSYZposvzdrdGrA90jO68SzBTHJ8A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=gmail.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=L1wQ1OlxyUNC6t/+EFekUqCIAmqr4wF1LuLBv5UCmok=; b=YB79CeklMcrzTuDGjDPmiHtNZK4MuUHvEZADu4jVcYgP517V32kXZqGsk3seOlNgev2tWwgV/aGrAl3oP5ul4qKq68OqTPdFViV8vDL8G7Jz23m5pnn83GKut4qTdpr+1+tWdQHei4f9kUtsUMMRuPNwtia8sHvXaqYKcFZzjA/FTBk7d6iRI7fjcY9Tz6DH4kZQqLH/MfggARL7HshAock54OjF93rFNdwFm9wMyUCkrfksO1SBqEJtW3FVHj5uUpkKhi0JFxbEf3dKKcaGYFnFdlNyWV3s4zs686eduol4ZbEHbMSk3vLe70XxkarNCnDPfBhzvvj7Cnf8Tu/GPg== Received: from DS7PR07CA0024.namprd07.prod.outlook.com (2603:10b6:5:3af::6) by PH7PR12MB7236.namprd12.prod.outlook.com (2603:10b6:510:207::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8422.12; Sat, 8 Feb 2025 09:03:38 +0000 Received: from DS3PEPF000099D5.namprd04.prod.outlook.com (2603:10b6:5:3af:cafe::bf) by DS7PR07CA0024.outlook.office365.com (2603:10b6:5:3af::6) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8398.24 via Frontend Transport; Sat, 8 Feb 2025 09:03:38 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by DS3PEPF000099D5.mail.protection.outlook.com (10.167.17.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8398.14 via Frontend Transport; Sat, 8 Feb 2025 09:03:38 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Sat, 8 Feb 2025 01:03:22 -0800 Received: from rnnvmail202.nvidia.com (10.129.68.7) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.14; Sat, 8 Feb 2025 01:03:22 -0800 Received: from Asurada-Nvidia.nvidia.com (10.127.8.12) by mail.nvidia.com (10.129.68.7) with Microsoft SMTP Server id 15.2.1544.14 via Frontend Transport; Sat, 8 Feb 2025 01:03:21 -0800 From: Nicolin Chen To: , , , CC: , , , , , , , , , , , , , Subject: [PATCH v1 13/13] iommufd/selftest: Add coverage for IOMMU_OPTION_SW_MSI_START/SIZE Date: Sat, 8 Feb 2025 01:02:46 -0800 Message-ID: <257ac74e767f29930aa47991224fab96f36c3899.1739005085.git.nicolinc@nvidia.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-NV-OnPremToCloud: AnonymousSubmission X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS3PEPF000099D5:EE_|PH7PR12MB7236:EE_ X-MS-Office365-Filtering-Correlation-Id: 68c58570-a574-4a3b-0efa-08dd481f79ea X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|82310400026|36860700013|7416014|1800799024; X-Microsoft-Antispam-Message-Info: ZsvpKbGLo0OursV4CLkUkdJL5llEiy/LwZXnOFpvLYzXG8hsYOGXz/9CzgxP3+0yqG7IdoSieCoKb5TB0U3ngCCnBkJ6MHxtBVAE/5tq+D3ceuTNgHXGNkAngIBnAErdQP3zlla3SgabemgHkwik9XSu/Fdwk50xA+9YqY/Tmz0AGa9yIjkMxaA+vrZu82fOgCyh/Er2CgspH6o3vCpxdyDeHwXaI2ZUNFmF+xi3L4tMaOMH+fCGCDle3yJl838ADe7Zi4Iioj9qTNPYG7tGk8g77VjJQwXRgbTHIvwK7KukDekYyBoUzu8Ffg9RSRExh6KQYsBQv51/PwVkzBUNAYAIz+l2NnFOUm2sluMfx82MVkU9rUPhuCX0M3EF8NQxwqLiQl9701CRg3UzKi7v5JIH8pE9p2rZsBAdOVa+VBjnJ1uDRXOj1nuj0ypVopVQni1p6OJ4fh3BLMaMF6ng7YiIBjU7+sTZ/7q36ZBYnXunQhvS3asZLtI+GY1x1a8mJUjTE27UdCdOGrj++CZYpzkq1daxHrnpMO63HCkXVyK/9Nqp7qfgDvtZ7xTzuYzxi6zGo1NNAuZLqVO0tFnH3BycAeTiMRPS5fhdj1DZVFc2rW4by5Cr7ULrNOA/CqrQ4VeDN02IWfPyDikIixxDe3jJcjtdhP+VYi9YNu7focSKOg3HMflRUuXdfTumInm5RRt1QgujSdrh2N2ATQAy7PGYlunQ8FMxUlc2RhUHMvZ531F3FRPj5i4FuNz2pUDODQz6ofn9W/k2U4OZlS9v3lBBXpccNAq0WdtWeNuqm+c//JLShiT0gCrfQ19WPlpDSRBAls6ToMdb6b4pFYVeUwd0OfAKmXsBps+I6bl5w08+0Bqmg0QMUMUr/tmAow3PDxT0Xqy2Hw2VSY2rOS7x+IdlKDM08IEupfTkn4GNiKlXkj5UtF0MG2prWewAgJSww+jUjHVvvTO69lwPjGAaUBJXc8kQ7tLlDufkvst1hFAo3mxCQcJKsEeJ+ANG0gb2Ux1hWvpBxbMj2RKY/2hDJpzcDVGWOcYIcloxNwxbUlhuWwdQE0kpS7ol2EGhgklPXNWACSwj+i4LawYi3P5CxFy/dp0wiXSMTK2OnFCMSKFy+BMXH8g3N5K1Zr6f+PnX9o0JUah6KGMcwD6zLElVhlbDGXqzUm6YhXXP1BxGWOhTZBcXDXk5JuAmJjKXZeDVuGotXdjPj/vgW2nwtE4Q6YD0ugtM9DRQ0IQYuqAaNrXNJEIcM1AcHksRgPPHLBGQtP+edLkPn3xdi7oOxBcaDV4DiUQzHhhB5dnQ4JRdY/3hlffOexyDAOPp6imf3tqfxJmzHcwhKq+9K9QRozizzvQbQNO3AvXdBNRcmm/cliO5jkRqnl3+PRK1UFq/7KKs08OYO7Wx+8OpLNih1x0PNCF2LqL5bKZW7M7lbXXmetgB8SzCHdjKjVRgI5x5bYOuzvQjqV4SzFTp7URwUTHJejSf9PI3/SrKmsi6E0eCZR4= X-Forefront-Antispam-Report: CIP:216.228.117.160; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge1.nvidia.com; CAT:NONE; SFS:(13230040)(376014)(82310400026)(36860700013)(7416014)(1800799024); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Feb 2025 09:03:38.1640 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 68c58570-a574-4a3b-0efa-08dd481f79ea X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.160]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DS3PEPF000099D5.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB7236 Also add fail_nth coverage too. Signed-off-by: Nicolin Chen --- tools/testing/selftests/iommu/iommufd.c | 97 +++++++++++++++++++ .../selftests/iommu/iommufd_fail_nth.c | 21 ++++ 2 files changed, 118 insertions(+) diff --git a/tools/testing/selftests/iommu/iommufd.c b/tools/testing/selftests/iommu/iommufd.c index a1b2b657999d..db7dfc5ae56a 100644 --- a/tools/testing/selftests/iommu/iommufd.c +++ b/tools/testing/selftests/iommu/iommufd.c @@ -334,6 +334,103 @@ TEST_F(change_process, basic) ASSERT_EQ(child, waitpid(child, NULL, 0)); } +FIXTURE(iommufd_sw_msi) +{ + int fd; + uint32_t ioas_id; + uint32_t idev_id[2]; +}; + +FIXTURE_SETUP(iommufd_sw_msi) +{ + self->fd = open("/dev/iommu", O_RDWR); + ASSERT_NE(-1, self->fd); + + test_ioctl_ioas_alloc(&self->ioas_id); + test_cmd_mock_domain(self->ioas_id, NULL, NULL, &self->idev_id[0]); + test_cmd_mock_domain_flags(self->ioas_id, MOCK_FLAGS_DEVICE_NO_ATTACH, + NULL, NULL, &self->idev_id[1]); +} + +FIXTURE_TEARDOWN(iommufd_sw_msi) +{ + teardown_iommufd(self->fd, _metadata); +} + +TEST_F(iommufd_sw_msi, basic) +{ + struct iommu_option cmd = { + .size = sizeof(cmd), + .op = IOMMU_OPTION_OP_SET, + }; + + /* Negative case: object_id must be a device id */ + cmd.object_id = self->ioas_id; + cmd.option_id = IOMMU_OPTION_SW_MSI_START; + cmd.val64 = 0x70000000; + EXPECT_ERRNO(ENOENT, ioctl(self->fd, IOMMU_OPTION, &cmd)); + cmd.object_id = 0; + cmd.option_id = IOMMU_OPTION_SW_MSI_SIZE; + cmd.val64 = 2; + EXPECT_ERRNO(ENOENT, ioctl(self->fd, IOMMU_OPTION, &cmd)); + + /* Negative case: device must not be attached already */ + if (self->idev_id[0]) { + cmd.object_id = self->idev_id[0]; + cmd.option_id = IOMMU_OPTION_SW_MSI_START; + cmd.val64 = 0x70000000; + EXPECT_ERRNO(EBUSY, ioctl(self->fd, IOMMU_OPTION, &cmd)); + } + + /* Device attached to nothing */ + if (self->idev_id[1]) { + /* Negative case: alignment failures */ + cmd.object_id = self->idev_id[1]; + cmd.option_id = IOMMU_OPTION_SW_MSI_START; + cmd.val64 = 0x7fffffff; + EXPECT_ERRNO(EINVAL, ioctl(self->fd, IOMMU_OPTION, &cmd)); + cmd.val64 = 0x7fffff00; + EXPECT_ERRNO(EINVAL, ioctl(self->fd, IOMMU_OPTION, &cmd)); + cmd.val64 = 0x7fff0000; + EXPECT_ERRNO(EINVAL, ioctl(self->fd, IOMMU_OPTION, &cmd)); + + /* Negative case: overlap against [0x80000000, 0x80ffffff] */ + cmd.option_id = IOMMU_OPTION_SW_MSI_START; + cmd.val64 = 0x80000000; + EXPECT_ERRNO(EADDRINUSE, ioctl(self->fd, IOMMU_OPTION, &cmd)); + cmd.val64 = 0x80400000; + EXPECT_ERRNO(EADDRINUSE, ioctl(self->fd, IOMMU_OPTION, &cmd)); + cmd.val64 = 0x80800000; + EXPECT_ERRNO(EADDRINUSE, ioctl(self->fd, IOMMU_OPTION, &cmd)); + cmd.val64 = 0x80c00000; + EXPECT_ERRNO(EADDRINUSE, ioctl(self->fd, IOMMU_OPTION, &cmd)); + /* Though an address that starts 1MB below will be okay ... */ + cmd.val64 = 0x7ff00000; + ASSERT_EQ(0, ioctl(self->fd, IOMMU_OPTION, &cmd)); + /* ... but not with a 2MB size */ + cmd.option_id = IOMMU_OPTION_SW_MSI_SIZE; + cmd.val64 = 2; + EXPECT_ERRNO(EADDRINUSE, ioctl(self->fd, IOMMU_OPTION, &cmd)); + + /* Set a safe 2MB window */ + cmd.option_id = IOMMU_OPTION_SW_MSI_START; + cmd.val64 = 0x70000000; + ASSERT_EQ(0, ioctl(self->fd, IOMMU_OPTION, &cmd)); + cmd.option_id = IOMMU_OPTION_SW_MSI_SIZE; + cmd.val64 = 2; + ASSERT_EQ(0, ioctl(self->fd, IOMMU_OPTION, &cmd)); + + /* Read them back to verify */ + cmd.op = IOMMU_OPTION_OP_GET; + cmd.option_id = IOMMU_OPTION_SW_MSI_START; + ASSERT_EQ(0, ioctl(self->fd, IOMMU_OPTION, &cmd)); + ASSERT_EQ(cmd.val64, 0x70000000); + cmd.option_id = IOMMU_OPTION_SW_MSI_SIZE; + ASSERT_EQ(0, ioctl(self->fd, IOMMU_OPTION, &cmd)); + ASSERT_EQ(cmd.val64, 2); + } +} + FIXTURE(iommufd_ioas) { int fd; diff --git a/tools/testing/selftests/iommu/iommufd_fail_nth.c b/tools/testing/selftests/iommu/iommufd_fail_nth.c index 64b1f8e1b0cf..be0d7735dfeb 100644 --- a/tools/testing/selftests/iommu/iommufd_fail_nth.c +++ b/tools/testing/selftests/iommu/iommufd_fail_nth.c @@ -615,6 +615,10 @@ TEST_FAIL_NTH(basic_fail_nth, access_pin_domain) /* device.c */ TEST_FAIL_NTH(basic_fail_nth, device) { + struct iommu_option cmd = { + .size = sizeof(cmd), + .op = IOMMU_OPTION_OP_SET, + }; struct iommu_hwpt_selftest data = { .iotlb = IOMMU_TEST_IOTLB_DEFAULT, }; @@ -624,6 +628,7 @@ TEST_FAIL_NTH(basic_fail_nth, device) uint32_t ioas_id; uint32_t ioas_id2; uint32_t stdev_id; + uint32_t idev_id2; uint32_t idev_id; uint32_t hwpt_id; uint32_t viommu_id; @@ -692,6 +697,22 @@ TEST_FAIL_NTH(basic_fail_nth, device) IOMMU_HWPT_DATA_SELFTEST, &data, sizeof(data))) return -1; + if (_test_cmd_mock_domain_flags(self->fd, ioas_id, + MOCK_FLAGS_DEVICE_NO_ATTACH, NULL, NULL, + &idev_id2)) + return -1; + + cmd.object_id = idev_id2; + cmd.option_id = IOMMU_OPTION_SW_MSI_START; + cmd.val64 = 0x70000000; + if (ioctl(self->fd, IOMMU_OPTION, &cmd)) + return -1; + + cmd.option_id = IOMMU_OPTION_SW_MSI_SIZE; + cmd.val64 = 2; + if (ioctl(self->fd, IOMMU_OPTION, &cmd)) + return -1; + return 0; }