From patchwork Tue Apr 8 16:53:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jason Gunthorpe X-Patchwork-Id: 880046 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2079.outbound.protection.outlook.com [40.107.94.79]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C3BE623DEAD; Tue, 8 Apr 2025 16:54:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.79 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131276; cv=fail; b=XPwtvS5YIPAoqj242GigJPzpOnxSm53whj8sgh2o9LuGC4jSyTO8JRZWp95zTr3aSXh3pM557qWID0/7mdmPU+gZFgEWhVGxI/lG3NbR0HcsN76cYr/H83fyLFhc+Vwi4HWin2KtI2a1H4L2GIgN/PACSepbHqn7yk8P7DKFmwo= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744131276; c=relaxed/simple; bh=EwUz2zpDl8ttuhnt8DzDNLoEp4n5kn4u/oKRixT9+cM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=rOttpVAWM3j3BEaeCtyf8GSqstwNXXX1AkUc/fFvUkgLlPqHiSUcKxUERuew/x5nG9+2afiRrfnwhyPWchn9PxBuSTtvI2m5VJmbtpwauKT73SxKV7YorIdOGhqaad09QCqIoQklpOtU8vkEs1C5frADxZxKn9sxYBY5KeyoZfs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=Kkr6nPb4; arc=fail smtp.client-ip=40.107.94.79 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="Kkr6nPb4" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=lEp3PcVo9RzWfkjGikgbJc4QsuFSRf09PIpZnd2kYKsV8LEL7St/O6FgPa2fQS74iX20wqIIBXYnyKk41oQermSnLSRje79dt5eDT6q9mRpvygHDeqGEsFAyJxiBV6Z+sV/kDsvFcE2AbtGO33KlDTF88EFDbjL8CqftAmNN7X6z1Hl65jx9Uw3QZW3KV4epQhGZ+or1IigbxYsL6Ive9NCYIeV9TYFLmPdMFIPhLXfv7trUPMpLIWBsXye9QFaVqSttEDIzwI2zNSJlP8iNvOPMFB+UcLANtb/9CxoYoU05Zthpy5HEmyhhWviPH6ImaJeieK7OM0H20D5xf3XsRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=SOMbIpYmrZnccYIV89P8MSgtHZzA8Ua/GI47Fo/WeoQ=; b=nbI9fgGHe1gPwzllkhDmo7t9ITqcCUtOR/xg/ptf+3SrRYQeW0oLCID0H6kx1Vy0sW864UvqIcyLClih2KUB9z9w64CVMDaHi6qJt6NRNtAyvNzGdwcerrqc5AhyRyyaaWMnDsbH1tCVCTox8ORv5Y+BIBsO6oymqxO3u7fwW35Vf9tACt/q7/DZTzuB6B/BfLWfEDmzzA++EpttqUwSie2GnFX1DZ7xlfN0v3lMvUkp1xLtqak3nIbDZfR2Mwzrjv6QmbB6xtrLoNKngcjNDRWXE3uHxFxMxgddKg8ybRRPLPMq/acvrY+hYvAdVqqXtRoD3lUz7Fu2VmEeI8Qpfw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=SOMbIpYmrZnccYIV89P8MSgtHZzA8Ua/GI47Fo/WeoQ=; b=Kkr6nPb4QkceM60udPp8rNFrBQogT3C+wn39MHm5DDbQjO6C1yIW7S2ubJcmQq3mcyB90zgZ3MumFhhMbGp+0757smGR2ZwWmPlxDp9atCj+ti/9Z9GALj/3N/f3FKmPCHG5klpYWr2IqfajDKyUkjK1y74STDzVaBx7xj1/dgTjMKtXLJUV39hwlI+LRaDu8VUUp6/zQ052cTn2EX7YGvw1akR2CaAevCgBOP3r//fTD8rb6KBSLLWfzynurh+xox0lkT7DmGyHMWxPN8DQvSA0SiEu18cDREZp5Irxaq5ukTJeeeQxdnVKZIePmbT8X1oTPS6Op9KFYpTSKuJzrQ== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by BL1PR12MB5971.namprd12.prod.outlook.com (2603:10b6:208:39a::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8606.34; Tue, 8 Apr 2025 16:54:22 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8606.028; Tue, 8 Apr 2025 16:54:22 +0000 From: Jason Gunthorpe To: Alexandre Ghiti , Alim Akhtar , Alyssa Rosenzweig , Albert Ou , asahi@lists.linux.dev, David Woodhouse , Heiko Stuebner , iommu@lists.linux.dev, Janne Grunau , Jernej Skrabec , Jonathan Hunter , Joerg Roedel , Krzysztof Kozlowski , linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-rockchip@lists.infradead.org, linux-samsung-soc@vger.kernel.org, linux-sunxi@lists.linux.dev, linux-tegra@vger.kernel.org, Marek Szyprowski , Neal Gompa , Palmer Dabbelt , Paul Walmsley , Robin Murphy , Samuel Holland , Suravee Suthikulpanit , Sven Peter , Thierry Reding , Tomasz Jeznach , Krishna Reddy , Chen-Yu Tsai , Will Deacon Cc: Alejandro Jimenez , Bagas Sanjaya , Lu Baolu , Joerg Roedel , Nicolin Chen , Pasha Tatashin , patches@lists.linux.dev, David Rientjes , Mostafa Saleh , Matthew Wilcox Subject: [PATCH v4 10/23] iommu/amd: Convert to use struct iommu_pages_list Date: Tue, 8 Apr 2025 13:53:58 -0300 Message-ID: <10-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> In-Reply-To: <0-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com> References: X-ClientProxiedBy: MN2PR15CA0050.namprd15.prod.outlook.com (2603:10b6:208:237::19) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) Precedence: bulk X-Mailing-List: linux-samsung-soc@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|BL1PR12MB5971:EE_ X-MS-Office365-Filtering-Correlation-Id: 6ff1420c-8e3d-4f6d-2093-08dd76bdff67 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|7416014|376014|366016|921020; X-Microsoft-Antispam-Message-Info: e1mgG12ZD9YD/OdZetbga5NPgflwgtOkf3nQILgl4sRE8OlNG38iVR5G6h7wma6ir+d+yHQY3BUxcprTibscw43KW7kBn/xzYQeKr1NlRvnA0Y5eUAEr1h2RPKSsOqNn0VA5b1YDMztUJKofqSLAtrtm07ClSObGgEiCxSKPXHWuD1ODlwYzCvnJ2kr9WEBdXqHzdFhNMPs3HEhhR4zPhCzlF3zvO4joNZMc1lOWCY008dXfwvyssNmsUD0JX7S9X5EvqcrWU9dYtMOFBt6bNByCPTq8z3/LtrmDN6/MZwJgALiXR5lqV76lWhPITEx6h9dkbZ3Qx3iBModAmoUmErMfYSYxtVseLTrAkd2TVUOqEZVVOE4x8NPe9GTFQQnTP3UpkQHVZWYpFbGHIZnyTpDb+EvrnVXIM2lOxZudTIidbp+fNIGk2sPD/EIrrO9ng2L5LiUMBkbN8DKlyv7e4U0it0exYUWeTTjCpiZb/lFtPqQrg6FpIc1vTnRJuycDO5PI5zLtP85p7SnoZhYkYm4TpJnOEqyGmVNpsp2532YwTQDdoLZUTGV7ZHULE8MM4inIUEOWIygOssNnG0xiTxXhcnPQczw6c1h6eouB/PfnMKX1aPPkGKopIIt6hiIxsJuEy6rdz6DcZ012sNaxirViuJPi9JeEXK+2lIbjc1cQKo6+1E5OBK3wMrZHLYBBUUB9Z6vNUfLcH2blU42+kHANzmosmEESk5KVrrYCU2ekH7Xj7aPPoUpoWsIY8bim0ZK8y9yGClkCDThBNord0LtTK/3UM8uy7by9vkN0OofFErUk7EkTPdqha2ZmRE9aNBppySYVDCdUW6lRiB2B34bNoRvbbDKX5C8fDVbRlpwhBFyfbuukmb9wAXwOqvOjwqKhpsw/Hd4/b68aKQnh5vS46lTVwnAmGBcgQSuuVtsusxQWTmRKLD9bznxdT5BP6+ivuq1U/c3n6SKOG5usS32juMtdzJ59zAeAlZQJ7OCKW4yYvJKp7ZnK+ZJ1CA/hRvdSTxvg3BnOP9JNoI5liL7ROT7SHNFKNsCyIWK59HxA6KCTLkn1CsxWtro8H14FEJ4nSRSJOsWtv1YHqIu088U1v0OQ1dRnU9YPC5TzwczmPxgk3J5BpNeFCO2KgxbUEdk4eqHP/tJLbtD8GSnSXh14kbKs5Yl2JXa6Yml4AcIiDgXpNRNpW/ypvCEPIHDiUEf4wuelYLaIWNXpWWpB6ONGighMb7adNZ+s+kdp7gkAmRDfsx68eOP01GlPUtb5HypuwBIhw6ymhbwhSxizWdiQDFR6nTlrqw4776pZ3xDUEPOnbZB71aFjFGv0RY6WKB0Vw5UVaFzokiyhqyUxmeJpV8gj2u5mIPYZMk5CghZ/w36sNLLuubcBx1WLsMPOl2KvOjOO2Ps1rp2h+R7rXq59Gc0pP1eQYbfc6alsJ5tBhBdiSq9irP4dtUzI0/lyWcianmN4RLr0L3FjVuKhTA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:CH3PR12MB8659.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(366016)(921020); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: gAVMuCMUc/bpqZ6e45CKzgedA5gHPpzqfgZjgUp217god5GZjkIqxutHiwi7HZoTzXw0PRaha+792TKoRzH9Y3HXcDP181NXJpibVE1PQm76X7RyuFcLOUp6pbVrVSYuI9ui2vtNl732J2fJaFei5M31EQ9RV8wMD+gzJe80HM1gfp5jbJK3SdoKRyZ+uOpgVhOu8U8SX713OQQ3f7xRDL/ght0D0Vos/lhraZ/G1kPMn5XBNGl+uTCblHe9Lk0azeMq1+TNDp3BCHTDDmbZYUmfSkhUqu2FV15GmH4bUPWXGoNBInOlqu69Ut5kOcARHkmsaKTT41dhDsG5GNZQnyCR/ZdNowblHN3QkBNtRrKZeqkWiOiqI+x3Z6FEVZ37mOuzMe2K5VmfZsCFtM83+Ke1lTwNO5YHsXuZRsiBMtOlMF1xVap0CE9sWnZs7Ys779LLniLyaAQ03QLBTm/4iVcy69Vu3HcazlKVbP8HXyBBYjnSPYEzGA5xt66C/vTzdhM+IjPlIQ4MALqi7qtYSdJf81dLEUCrpUFtooIppF9bZMi4F5LnYMZKj8/TrVtl0e9s1lGLf8nTeHvd2iMyfy2JkbsR8TEEWj+APhRYxJguZ4Dn4DcH0sbU/GGDQn/VSYgdKvznks7g2jPcUFzFeylfuRZt9/sdtZpIoTHvfIQx0fQAB5Yt1gu8ewWjE+vMJt6ucwi5Krs+VpzcGhIgenTqw3fxa+HZ+S2cf4b3JYUXYH8bj2u5ymvOhDFCV38LFjeN+NHyEhoi1lE/Uuf+CfAxjTplcY1GkqxTyx+9SwGkV9nhavrSPrpnlgJ2cFhvNhoVqzfeJ11gs/M2sY9Qa98acGk7T8rUTiEghy54HbMP8WfncHaAyEwfJ17vLuCrPuaGL325WPouoemsf9edm5QX+h00YcMbOzgXUAWqzywlBDW19pxXLrbMpTIf3KMD9uip16jsdSzUlDi5yD4cEMhdRi7z8xdZe+9qbc3xq1kcj2+xDB/+qoHHd/YW629MLsAqoAJfrRjzLe57ahbkpIuYuJ6CxQgah/DT/x0CutGFW3iIIg6Qa1JV/ncpn9jwSRUwUyovHqzyidi/pKvIgApLgaIC13bgP+ovMWj9vNbt7nYGnAPipNDE5MDs6P6w64ssjWuSr/Y1N2QouJN0FZJ9u+bhBanMOtQ1dMYzm5IyYziO2h1l4kcp0JL+DSVvGip0DBUALPl72rwRis7ihsg25C/7omHYfbla+Mlg7RKCCs9P2fpTXsP1MRg3uYQPdc8hfbO4sK0ho50nynDJ/rNJC8qrGqxCjkwzovcKYI74lUuZNlXN32gbuvfEisILsmsiak9Ffvg472eq8s67AftI+F2YQPs5Ly0a/7w32rS1G09WV99mnAsOyh8oYGyocvA37fOV5qzTITNIptXPIBRe7a4o/hJhdxJbwGdumTfIiwZVagy/d85Rdmfde27XJR0yCnz9PuWB0K/XRYoo3wEkRV23DwC5GgKnKzmUBDrxmjPTWv1AGEF2/lQIDOedO1K4f6B8Le+XLh5ipUyVu6rAXWLbQMNPQc6iDi40DMs4yy4zNQpexI4gYOXR52Og X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6ff1420c-8e3d-4f6d-2093-08dd76bdff67 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2025 16:54:16.5679 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: IYmXri0hG9r4hYfkkoFv0T2JyouWGIW29bAKuKTngD4Y4T+vy7aeJHmgr4SgGUqK X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5971 Change the internal freelist to use struct iommu_pages_list. AMD uses the freelist to batch free the entire table during domain destruction, and to replace table levels with leafs during map. Tested-by: Alejandro Jimenez Signed-off-by: Jason Gunthorpe --- drivers/iommu/amd/io_pgtable.c | 26 ++++++++++---------------- 1 file changed, 10 insertions(+), 16 deletions(-) diff --git a/drivers/iommu/amd/io_pgtable.c b/drivers/iommu/amd/io_pgtable.c index 42d9e6746dee28..97ddadbcfb54d6 100644 --- a/drivers/iommu/amd/io_pgtable.c +++ b/drivers/iommu/amd/io_pgtable.c @@ -47,14 +47,7 @@ static u64 *first_pte_l7(u64 *pte, unsigned long *page_size, return fpte; } -static void free_pt_page(u64 *pt, struct list_head *freelist) -{ - struct page *p = virt_to_page(pt); - - list_add_tail(&p->lru, freelist); -} - -static void free_pt_lvl(u64 *pt, struct list_head *freelist, int lvl) +static void free_pt_lvl(u64 *pt, struct iommu_pages_list *freelist, int lvl) { u64 *p; int i; @@ -77,20 +70,20 @@ static void free_pt_lvl(u64 *pt, struct list_head *freelist, int lvl) if (lvl > 2) free_pt_lvl(p, freelist, lvl - 1); else - free_pt_page(p, freelist); + iommu_pages_list_add(freelist, p); } - free_pt_page(pt, freelist); + iommu_pages_list_add(freelist, pt); } -static void free_sub_pt(u64 *root, int mode, struct list_head *freelist) +static void free_sub_pt(u64 *root, int mode, struct iommu_pages_list *freelist) { switch (mode) { case PAGE_MODE_NONE: case PAGE_MODE_7_LEVEL: break; case PAGE_MODE_1_LEVEL: - free_pt_page(root, freelist); + iommu_pages_list_add(freelist, root); break; case PAGE_MODE_2_LEVEL: case PAGE_MODE_3_LEVEL: @@ -299,7 +292,8 @@ static u64 *fetch_pte(struct amd_io_pgtable *pgtable, return pte; } -static void free_clear_pte(u64 *pte, u64 pteval, struct list_head *freelist) +static void free_clear_pte(u64 *pte, u64 pteval, + struct iommu_pages_list *freelist) { u64 *pt; int mode; @@ -328,7 +322,7 @@ static int iommu_v1_map_pages(struct io_pgtable_ops *ops, unsigned long iova, int prot, gfp_t gfp, size_t *mapped) { struct amd_io_pgtable *pgtable = io_pgtable_ops_to_data(ops); - LIST_HEAD(freelist); + struct iommu_pages_list freelist = IOMMU_PAGES_LIST_INIT(freelist); bool updated = false; u64 __pte, *pte; int ret, i, count; @@ -353,7 +347,7 @@ static int iommu_v1_map_pages(struct io_pgtable_ops *ops, unsigned long iova, for (i = 0; i < count; ++i) free_clear_pte(&pte[i], pte[i], &freelist); - if (!list_empty(&freelist)) + if (!iommu_pages_list_empty(&freelist)) updated = true; if (count > 1) { @@ -524,7 +518,7 @@ static int iommu_v1_read_and_clear_dirty(struct io_pgtable_ops *ops, static void v1_free_pgtable(struct io_pgtable *iop) { struct amd_io_pgtable *pgtable = container_of(iop, struct amd_io_pgtable, pgtbl); - LIST_HEAD(freelist); + struct iommu_pages_list freelist = IOMMU_PAGES_LIST_INIT(freelist); if (pgtable->mode == PAGE_MODE_NONE) return;