From patchwork Thu Apr 3 18:33:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Terry Bowman X-Patchwork-Id: 878098 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2064.outbound.protection.outlook.com [40.107.96.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 64F3E253358; Thu, 3 Apr 2025 18:33:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.96.64 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743705221; cv=fail; b=STaYhY2nOMwi56+Plf96RSN1ltfHwRJ882iti4hl3LiE3RB9coyjw1cnOEVpHU3rSBghL61M5do8AGFeF3nXIIiqgU5SgH64vzJJ3zZ14M6LhH0LMHdWBH4SdWgsFG9GSvattheP8Eehbmb69V86ZqFIohQGYdjE4WD91Q2bNjU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743705221; c=relaxed/simple; bh=m+6V5bZBytlD9zruM+9+jBppyxSolXw2iFWcfq1D0IE=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=PwoQxJpqh85rlUlNegDmrEdiX9rWNGOIfw1Lo0WvtI/kjfmR37hD1IFK5NuS4nerF+cGkkp+l1VEhPqpUJR+UL/TPIllNYbX72OYrsYQBa1almtnADPCtMhXvvJjRLj+QagWlVGeWzQF5qABEZZxHEGJmXdsWEvutfqZcJLCIKg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=f+kRGhki; arc=fail smtp.client-ip=40.107.96.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="f+kRGhki" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=N8BKm+fQZ+wHhxWl3olHrQZs+CaHowcDT4DuLgKLz2CFccnvG4zzQgGakpnbwz0JWzfZixCkGcewCtUCoDkYy0XxrtTo9ugmA1GCJ7/f+M7MnqFzfG3+TJY0DA0fEyi1RJ4dbXvoCVE3CZ/X0VZsqObKOxr9hY9t6W802Xei23YKltFi5kLiLT8jaQXRpo3X59AxbTVDH9Ck1r8HSERnyDAqTjpjDP8frGqZgFg3S7rw8TNnOdA7ljP1UhWjxxrVCmAp1QsTSBIxN8irKmgMWGIxQ7RrZTJkEUjoghqk6flfs+KecS+ZwAWFhJwiDvoinPOX5mq9RsADzx++dxDSEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=KiDKEEAmDjq6v2zb3gdKuFidXF7+aJSnulzAIwD91sw=; b=OYZIsE3VvYm3cEH9Wrn4F3d7RkMGhKLxmP1+XXZO9wbe9VEzx+OK/ACMxKfolm1EVAr81IK1OYb7ifjp/YHBQSO7HNHD2zG0nL9B68/M/3q0KWYyz73V9yK6VdRiLPXdfPf3q0C5tGeDnmoqLZA+wJlcp4lmI8trR2Rm3NvkK2Bo0nSQhzY+fKgZiddc30LKp6sd7RIRYLAkihhIxfdI+AMpfQ6fJbFenqSemr6xYhnmpPTLT4JNzCDWDuzNP78KiZ0NDrbwBSAFBf/4ZHFJkW2/KS8b4uBKQQ0pqkxjkS59JcWk9HsjC+dUwkdyu+msI5asFXfqLL9O59dNN//XBQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=KiDKEEAmDjq6v2zb3gdKuFidXF7+aJSnulzAIwD91sw=; b=f+kRGhkip0pa9FGL4rYAGOCWUOGVt/8jJXBrwT2zbz6aOtBOh1fjVvgzXZUhJf9YC4sNVzq845zWEJy87eOf9PwPR8Jo4xb/3PlkZw1AvtBQ/TpKjwxGggpylL9+2lKsQSShRROVa9QmKiDX9D2z3jguXVTThlL/UYqXpKzLdug= Received: from MW4PR04CA0107.namprd04.prod.outlook.com (2603:10b6:303:83::22) by CH3PR12MB9077.namprd12.prod.outlook.com (2603:10b6:610:1a2::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.49; Thu, 3 Apr 2025 18:33:35 +0000 Received: from MWH0EPF000971E4.namprd02.prod.outlook.com (2603:10b6:303:83:cafe::6b) by MW4PR04CA0107.outlook.office365.com (2603:10b6:303:83::22) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8606.27 via Frontend Transport; Thu, 3 Apr 2025 18:33:34 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MWH0EPF000971E4.mail.protection.outlook.com (10.167.243.72) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8606.22 via Frontend Transport; Thu, 3 Apr 2025 18:33:34 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Thu, 3 Apr 2025 13:33:33 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v3 1/4] kernel/resource: Provide mem region release for SOFT RESERVES Date: Thu, 3 Apr 2025 13:33:12 -0500 Message-ID: <20250403183315.286710-2-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250403183315.286710-1-terry.bowman@amd.com> References: <20250403183315.286710-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000971E4:EE_|CH3PR12MB9077:EE_ X-MS-Office365-Filtering-Correlation-Id: c1ded528-fa0d-401b-4fdc-08dd72de0af9 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|1800799024|7416014|376014|36860700013|82310400026|921020; X-Microsoft-Antispam-Message-Info: xbe1tUGv8Evpp1RprCZqTA8d+a2Pc8paReA1jPFJBx7190AXZQUEbeHQI5Gv6IsDwwKZzjxH1pRfjMDR7sGq/PMvx71ehXl8bXzZZVksu8mGXXs9RGUo4D+m7kzigrV7Zh9l6gY/IUfv683dvfqQ3n8dvepLbRqWiWVGDAPsbsjxRMkjDSRqMVmWB+SMHJ+ya2sTQV8MrqATnJLGDBF2KV7kHVDhSN/EMMvXnnLuqAXv2pxRsNVzvLT6RqJzW804Xr0nlabpgNvq3Jnzdj+SOkEAJgP1X1UY5wKBCruCgcsI8P7+WY145jIVFfExaRMV+Qr9gF6t0sRoig3+iY4zD5HM+eQk9+e1Agr+d9Z5ylmhHXCyhc1eYvGd+CI4s4VgmjowIOn8fA8PQU2aI/BOUZ/x+htAi/Ha+lyw5a5o2ObsA+S/Y4jiXIpW+GnKNSK4WKXBZJ9SXjj+nt7fAX4AFEZ1p9OrBnZLLrIAdOOn3WUtfXZ6KEhmguvx4OcxGmvC21HZIgXMUO6LgD2sXoMMXKJFAuc4mDaIV/iIXsDAwfJKp4/RL7uLD0guPwt5YcO1IR9oBwyw7RXAA5LZjRh49KBqYOmNVlenk0tt1GyPu9DnUemNQCHEk4tPa/IqOwxAx/kcuHdTnEe/7DoOhxzBDI4SalR852NijE3yWst8XRwgzuB9kt0+Dg9MsXbL8vZBVBf7WN9cUzgTw+kf2jB3FHh1iS887JjVydK9CIRkm9Y7GrZA4Sg7l8qPoY28QOvpL+iwo+F7wzXYcfcx/46aBpvO383Vi752EKra9ShDwt50FEjZZ6v3enJTta5ufkFTDA4ElDKUm9/bxnu9UIzoPHL1zQgS5lwlr7c18t85HLilWK6vJWlkSuRN18L3ifTegQpYx7mxeGfri15pjYbb1g9XYAAraKrpPSqRSa2XdjbtI3KReKe0jeKv94EgY9+FInaFVxiKP9XrGczmFGX4KPM7/0JDfMEhQYZq2BiAXhrMjavyA01cAhNw8dOFiJmvyscMSeuhZb85hHP/dUXmG+q484tBaexmGa2XI5v9OhJUGtMf/gj2SDMZKnj0Ci1WgXmffKAqKjtrF9nnzIqe5bC/3/Mr68VykHogGd7G5Ksc0I3EHelUnKA8Yvb7theb2WxMJPobl4FWTNV6P+DTvfiCRyn3hlW0T2bCgbDbZIPP9YPIxUZ1c0pcf+eUrt/+aYFaw8Gc3y0Of5YbF6CqGMUyDKXL1fctjKOLRc23gwhAlTsI5OAOrqSEGH57XSp22pMJ55EIQFBezEm9Rbv9xNPCsMTJIM1hXOM0aoTRspecNlpNBy7KquqkiZK1hoNb39hXTffLKfQrwjKN1nimjVmwQ6WXo354S99YJWHLtGKxQ1alEBBCr9CdZ7As+fm/kbt/rgqwD8o98O9YQEITch8wyCP8wqQQ1cHwejyGDjiwn8n2xCbyaLvEF8SYZ16YKlKn0Oe9H4VhynXdP1UETwG35ym7i2UeUMycQqV5GbM= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(1800799024)(7416014)(376014)(36860700013)(82310400026)(921020); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2025 18:33:34.7489 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: c1ded528-fa0d-401b-4fdc-08dd72de0af9 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000971E4.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9077 From: Nathan Fontenot Add a release_Sam_region_adjustable() interface to allow for removing SOFT RESERVE memory resources. This extracts out the code to remove a mem region into a common __release_mem_region_adjustable() routine, this routine takes additional parameters of an IORES descriptor type to add checks for IORES_DESC_* and a flag to check for IORESOURCE_BUSY to control it's behavior. The existing release_mem_region_adjustable() is a front end to the common code and a new release_srmem_region_adjustable() is added to release SOFT RESERVE resources. Signed-off-by: Nathan Fontenot Signed-off-by: Terry Bowman --- include/linux/ioport.h | 3 +++ kernel/resource.c | 55 +++++++++++++++++++++++++++++++++++++++--- 2 files changed, 54 insertions(+), 4 deletions(-) diff --git a/include/linux/ioport.h b/include/linux/ioport.h index 5385349f0b8a..718360c9c724 100644 --- a/include/linux/ioport.h +++ b/include/linux/ioport.h @@ -357,6 +357,9 @@ extern void __release_region(struct resource *, resource_size_t, #ifdef CONFIG_MEMORY_HOTREMOVE extern void release_mem_region_adjustable(resource_size_t, resource_size_t); #endif +#ifdef CONFIG_CXL_REGION +extern void release_srmem_region_adjustable(resource_size_t, resource_size_t); +#endif #ifdef CONFIG_MEMORY_HOTPLUG extern void merge_system_ram_resource(struct resource *res); #endif diff --git a/kernel/resource.c b/kernel/resource.c index 12004452d999..0195b31064b0 100644 --- a/kernel/resource.c +++ b/kernel/resource.c @@ -1387,7 +1387,7 @@ void __release_region(struct resource *parent, resource_size_t start, } EXPORT_SYMBOL(__release_region); -#ifdef CONFIG_MEMORY_HOTREMOVE +#if defined(CONFIG_MEMORY_HOTREMOVE) || defined(CONFIG_CXL_REGION) /** * release_mem_region_adjustable - release a previously reserved memory region * @start: resource start address @@ -1407,7 +1407,10 @@ EXPORT_SYMBOL(__release_region); * assumes that all children remain in the lower address entry for * simplicity. Enhance this logic when necessary. */ -void release_mem_region_adjustable(resource_size_t start, resource_size_t size) +static void __release_mem_region_adjustable(resource_size_t start, + resource_size_t size, + bool busy_check, + int res_desc) { struct resource *parent = &iomem_resource; struct resource *new_res = NULL; @@ -1446,7 +1449,12 @@ void release_mem_region_adjustable(resource_size_t start, resource_size_t size) if (!(res->flags & IORESOURCE_MEM)) break; - if (!(res->flags & IORESOURCE_BUSY)) { + if (busy_check && !(res->flags & IORESOURCE_BUSY)) { + p = &res->child; + continue; + } + + if (res_desc != IORES_DESC_NONE && res->desc != res_desc) { p = &res->child; continue; } @@ -1496,7 +1504,46 @@ void release_mem_region_adjustable(resource_size_t start, resource_size_t size) write_unlock(&resource_lock); free_resource(new_res); } -#endif /* CONFIG_MEMORY_HOTREMOVE */ +#endif + +#ifdef CONFIG_MEMORY_HOTREMOVE +/** + * release_mem_region_adjustable - release a previously reserved memory region + * @start: resource start address + * @size: resource region size + * + * This interface is intended for memory hot-delete. The requested region + * is released from a currently busy memory resource. The requested region + * must either match exactly or fit into a single busy resource entry. In + * the latter case, the remaining resource is adjusted accordingly. + * Existing children of the busy memory resource must be immutable in the + * request. + * + * Note: + * - Additional release conditions, such as overlapping region, can be + * supported after they are confirmed as valid cases. + * - When a busy memory resource gets split into two entries, the code + * assumes that all children remain in the lower address entry for + * simplicity. Enhance this logic when necessary. + */ +void release_mem_region_adjustable(resource_size_t start, resource_size_t size) +{ + return __release_mem_region_adjustable(start, size, + true, IORES_DESC_NONE); +} +EXPORT_SYMBOL(release_mem_region_adjustable); +#endif + +#ifdef CONFIG_CXL_REGION +void release_srmem_region_adjustable(resource_size_t start, + resource_size_t size) +{ + return __release_mem_region_adjustable(start, size, + false, IORES_DESC_SOFT_RESERVED); +} +EXPORT_SYMBOL(release_srmem_region_adjustable); +#endif + #ifdef CONFIG_MEMORY_HOTPLUG static bool system_ram_resources_mergeable(struct resource *r1, From patchwork Thu Apr 3 18:33:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Terry Bowman X-Patchwork-Id: 878541 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2055.outbound.protection.outlook.com [40.107.94.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A780253333; Thu, 3 Apr 2025 18:33:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.94.55 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743705232; cv=fail; b=ufBO3oCgZKEuPkgYZkebYe8E6Cye01mCk53IUgktCCdmcBUbKh2VKLuGHMqrh2bA9JZMeuUYK3D6qqkVjeRWCNIDd73F9oQxzG29RmVPwz9CYzMbXW/HX3f314x60fKQEfKagEoKuPSsdJ/kOfynFJD2aWp3eP51cPhNmOYNM4o= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743705232; c=relaxed/simple; bh=7enGw8AteUg2sETSeRZeRLxvS7FbRpzXPKyfn4nbYAs=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=pskgt7Q5ZRSw6ngMlimTybcIod9K8K8c52b480roZZW+fUcRXRMoOgofh73TXGcMpSYUvYUNPu4uQih+S68Hj2/iBSAMUyu85bPiFwVOWYSsuAg8XFjbp647J7cWfOB8cl7BPxqtGyg0LwLLqGmHZ3htWDGhXVcitneOj3y+SZs= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=Y9lcFCy9; arc=fail smtp.client-ip=40.107.94.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="Y9lcFCy9" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=JFKq5PkOEJEe3jrnjZ3wVsaEZLXayDAkZnu3M93Vf9yytSa1WZLniEfLm0rbMHk44GARLzshEI62cFicEm0yRtne+RGZ0Le1siG8M5BsJ47a1GCn1qP3qRWT1f/5g53D8hPGyuRofWrJFfwquuUJDrlyzuY97bpYSaTTLQg2zsCDuM9c3Vp1dgk1ffCMDkgyJJXa5/q68ASpTkBIiOg41VDk9EvzD0y5PlpGoO5UvugaCAGSHmu0+WE/Eq2sOEPJfspnqYvcCI+wY2y9cSuReYIcOxYcaNuAUmH8VUM6hzhFK53T/0EmkkOuXH1FHNyTjKEd9DxO2GX6EFTE4uQWOQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=iV3bDj45tW9zU77rgwTVWgJQ1onwWYeBqOk/WtcpY2U=; b=mL2IG4w3afbuvvDDnm2adlQ9HeOxcFBgqdLcl2SAdqhYFODCKs4cL5eevgV/4v67hJAcWMZ4mH/zE+PdtaT+9ik+b2JxCiyFL/NqadA+nH8fK36L/siW7TANXWoSlPHBGI+bUrJ3kxZTGftdPh5/w/99u29aQl0Waq+QHHSjGk4lbsviSoVDR2d8sjXDKrXFubUDoK+0+/qq0shWkbmBvIic2v4eIubf2uxtqcjk5/A5TnhNbItEtqNRkCM7wNUezjh5k3I6nPSEU4/wC4Q+M0DuD6edWwnDMIojGUX3fJ7PBahvi+NVrokMDRPA7sDvixyQfgupqa4DlWBMxMvP5A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=iV3bDj45tW9zU77rgwTVWgJQ1onwWYeBqOk/WtcpY2U=; b=Y9lcFCy9MFyhCSdByOi2F0ZsJnZpqMJav8Q2wamfUCNPnd0u9h+/hypcxWt2Nrf44fWNCu5jOCqHGI5gAAJkO8jhpwW0ByjaHHy2QSHwRCC7+hRXIrDlxy1mfdGWuET0K5HI8XHzIS0SAR1+Nk1fJKjmrF7slx+4qvy/4YaBkks= Received: from SJ0PR03CA0102.namprd03.prod.outlook.com (2603:10b6:a03:333::17) by DS0PR12MB8573.namprd12.prod.outlook.com (2603:10b6:8:162::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8583.41; Thu, 3 Apr 2025 18:33:46 +0000 Received: from MWH0EPF000971E6.namprd02.prod.outlook.com (2603:10b6:a03:333:cafe::19) by SJ0PR03CA0102.outlook.office365.com (2603:10b6:a03:333::17) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.52 via Frontend Transport; Thu, 3 Apr 2025 18:33:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MWH0EPF000971E6.mail.protection.outlook.com (10.167.243.74) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8606.22 via Frontend Transport; Thu, 3 Apr 2025 18:33:46 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Thu, 3 Apr 2025 13:33:44 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v3 2/4] cxl: Update Soft Reserved resources upon region creation Date: Thu, 3 Apr 2025 13:33:13 -0500 Message-ID: <20250403183315.286710-3-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250403183315.286710-1-terry.bowman@amd.com> References: <20250403183315.286710-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000971E6:EE_|DS0PR12MB8573:EE_ X-MS-Office365-Filtering-Correlation-Id: 0ffe017b-c73f-4b42-4e8f-08dd72de11d6 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|7416014|36860700013|82310400026|1800799024|921020; X-Microsoft-Antispam-Message-Info: 9EC1C8IsaMaICHcgmaPQ3/KCDwuTHJ1XOJCZFZu3aL22J8NuSKTd1BBGEIaFH9O7UWe8xmJOxr5UbLfEgmVuJ4t8iawhVxQ/sIY+6/slos6frFcsPO/ZdLUX3IDElYEsCTYwD+J+SjwJl4/BrAwpro++U3HQuZ/Ivpqw62nzX/JS/rnGb+Jzso33bvSufVi9kVmgiP/ro8tQUQUkb6kMZT+Wo/ZzYkVZwqWz1ApEAD7y0QbBsT4sB+EkkBFmn06X6quH6GllZWDWQOfdcqMAN/bXZ2hCGf17C8fAIuy2RhRROtzrdcmcy8Pq3YLedq+aTF3HCi5NsnF+ZPMQy2YcvMH8EODbPqpTfUaIJULd1yCE9qz1FII2+ssLPoko4chtKt/ZI44ahwBdw0ou46aBdd8Ctk9zMutauuQZPBYLHrVF59FWvKXNljDkcDEQTkZ0vlP4P6AL/PTZcNVHL/9zxbPXapOVzn9RJFk1zSTyVr4i/AdFGQP6dba1doQNmOsxtUi2cnDGM7ivdPb+rntNTnf+IkfIoQdy1Pu7NaGBydxhJxBgxAowsM2vV8KDal4UEGPMbVCL3ceIwwWhKvQI3x6I4KCh0vctHIIEwSrAwChT9f1UyNsz6pLH0PNEIEeJoq6jOOl6MQWd4izBQi26XyZKEpmd1W13qGEdRbqNmeXhqEeUDZuPUydOpi8IUy2NuMkdUzgoV5vX2+VEAusxYRbDMvp2qNOtC3NuYvQrx13dqAeLq4vDerzBJR18c5T1j39DLe/OppHVBlxSow9KzT0BU3D5n50QqGHD+60W52pEcMJAlQds2AuFk1KrTlQJi6dBVnyPxz/9z8qf1rxUCW4OiYltH0rFP3PTxNs6tQDP2isUj8gKlII7/myXWacOFcbC1oKDk8sFf+DZAKoiPARz4fXLQKDu1FqW5sMoLrb27MXWSgijneudzzpI7NEcmOoNOlBxd3G4XwZ/i5E3DGxngQpVXEu3jNUaJXMJb6MvISFZRaofwljAm+MTxjcPpw8PhgebZz3vZJxn8n3fM5kzpFL7PYBnb9zEVOu456JYWP8fjanYqpk3ztk6flrxeL+acl+wbb4iKCSRoevHlhG1oftEotk2ZP/sIk9DFr9vs+XYwZcq3ukShEcL7GHQXa0ThNop6jmw9T6sYQQG1gu7U1nlpygIhnKfhETAft6JiGCA24/QXHB9Rn1t/0hROX6TAtoNIixm+Rh1caj9CxZr4+JyPonOQpwqngbQGdnFhlx08A8SL/fqVzX8mHjaxEhydM0rEpeDm50AYyywWKZOEVXFhYAd0yPqN6rh6CWNcQ4/ank6O3jrEi7xKV4fjwgXXlX0IGPGNd0vBWv18e6DjlKH1Ai1t4TlwGcx/PLIblneT0SJdJNQ+EFEt/sbtbT7RCQyabzX7Y4fInlyDtfSw67i7YnztVWh6iYgYeeMBMI8ejrYujwFNpciv+w81+9eI1v0WIBM6a4EHy/jRtWYKNLGK1rPiTOVFG4Drcc= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(376014)(7416014)(36860700013)(82310400026)(1800799024)(921020); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2025 18:33:46.2635 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0ffe017b-c73f-4b42-4e8f-08dd72de11d6 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000971E6.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8573 From: Nathan Fontenot Update handling of SOFT RESERVE iomem resources that intersect with CXL region resources to remove intersections from the SOFT RESERVE resources. The current approach of leaving SOFT RESERVE resources as is can cause failures during hotplug replace of CXL devices because the resource is not available for reuse after teardown of the CXL device. To accomplish this the cxl acpi driver creates a worker thread at the end of cxl_acpi_probe(). This worker thread first waits for the CXL PCI CXL mem drivers have loaded. The cxl core/suspend.c code is updated to add a pci_loaded variable, in addition to the mem_active variable, that is updated when the pci driver loads. Remove CONFIG_CXL_SUSPEND Kconfig as it is no longer needed. A new cxl_wait_for_pci_mem() routine uses a waitqueue for both these driver to be loaded. The need to add this additional waitqueue is ensure the CXL PCI and CXL mem drivers have loaded before we wait for their probe, without it the cxl acpi probe worker thread calls wait_for_device_probe() before these drivers are loaded. After the CXL PCI and CXL mem drivers load the cxl acpi worker thread uses wait_for_device_probe() to ensure device probe routines have completed. Once probe completes and regions have been created, find all cxl regions that have been created and trim any SOFT RESERVE resources that intersect with the region. Update cxl_acpi_exit() to cancel pending waitqueue work. Signed-off-by: Nathan Fontenot Signed-off-by: Terry Bowman --- drivers/cxl/Kconfig | 4 ---- drivers/cxl/acpi.c | 28 ++++++++++++++++++++++++++ drivers/cxl/core/Makefile | 2 +- drivers/cxl/core/region.c | 24 +++++++++++++++++++++- drivers/cxl/core/suspend.c | 41 ++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxl.h | 3 +++ drivers/cxl/cxlmem.h | 9 --------- drivers/cxl/cxlpci.h | 1 + drivers/cxl/pci.c | 2 ++ include/linux/pm.h | 7 ------- 10 files changed, 99 insertions(+), 22 deletions(-) diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig index 205547e5543a..c7377956c1d5 100644 --- a/drivers/cxl/Kconfig +++ b/drivers/cxl/Kconfig @@ -117,10 +117,6 @@ config CXL_PORT default CXL_BUS tristate -config CXL_SUSPEND - def_bool y - depends on SUSPEND && CXL_MEM - config CXL_REGION bool "CXL: Region Support" default CXL_BUS diff --git a/drivers/cxl/acpi.c b/drivers/cxl/acpi.c index cb14829bb9be..94f2d649bb30 100644 --- a/drivers/cxl/acpi.c +++ b/drivers/cxl/acpi.c @@ -7,6 +7,8 @@ #include #include #include +#include +#include #include #include "cxlpci.h" #include "cxl.h" @@ -813,6 +815,27 @@ static int pair_cxl_resource(struct device *dev, void *data) return 0; } +static void cxl_srmem_work_fn(struct work_struct *work) +{ + /* Wait for CXL PCI and mem drivers to load */ + cxl_wait_for_pci_mem(); + + /* + * Once the CXL PCI and mem drivers have loaded wait + * for the driver probe routines to complete. + */ + wait_for_device_probe(); + + cxl_region_srmem_update(); +} + +DECLARE_WORK(cxl_sr_work, cxl_srmem_work_fn); + +static void cxl_srmem_update(void) +{ + schedule_work(&cxl_sr_work); +} + static int cxl_acpi_probe(struct platform_device *pdev) { int rc; @@ -887,6 +910,10 @@ static int cxl_acpi_probe(struct platform_device *pdev) /* In case PCI is scanned before ACPI re-trigger memdev attach */ cxl_bus_rescan(); + + /* Update SOFT RESERVED resources that intersect with CXL regions */ + cxl_srmem_update(); + return 0; } @@ -918,6 +945,7 @@ static int __init cxl_acpi_init(void) static void __exit cxl_acpi_exit(void) { + cancel_work_sync(&cxl_sr_work); platform_driver_unregister(&cxl_acpi_driver); cxl_bus_drain(); } diff --git a/drivers/cxl/core/Makefile b/drivers/cxl/core/Makefile index 086df97a0fcf..035864db8a32 100644 --- a/drivers/cxl/core/Makefile +++ b/drivers/cxl/core/Makefile @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_CXL_BUS) += cxl_core.o -obj-$(CONFIG_CXL_SUSPEND) += suspend.o +obj-y += suspend.o ccflags-y += -I$(srctree)/drivers/cxl CFLAGS_trace.o = -DTRACE_INCLUDE_PATH=. -I$(src) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index c3f4dc244df7..25d70175f204 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include "core.h" @@ -2333,7 +2334,7 @@ const struct device_type cxl_region_type = { bool is_cxl_region(struct device *dev) { - return dev->type == &cxl_region_type; + return dev && dev->type == &cxl_region_type; } EXPORT_SYMBOL_NS_GPL(is_cxl_region, "CXL"); @@ -3443,6 +3444,27 @@ int cxl_add_to_region(struct cxl_port *root, struct cxl_endpoint_decoder *cxled) } EXPORT_SYMBOL_NS_GPL(cxl_add_to_region, "CXL"); +int cxl_region_srmem_update(void) +{ + struct device *dev = NULL; + struct cxl_region *cxlr; + struct resource *res; + + do { + dev = bus_find_next_device(&cxl_bus_type, dev); + if (is_cxl_region(dev)) { + cxlr = to_cxl_region(dev); + res = cxlr->params.res; + release_srmem_region_adjustable(res->start, + resource_size(res)); + } + put_device(dev); + } while (dev); + + return 0; +} +EXPORT_SYMBOL_NS_GPL(cxl_region_srmem_update, "CXL"); + u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa) { struct cxl_region_ref *iter; diff --git a/drivers/cxl/core/suspend.c b/drivers/cxl/core/suspend.c index 29aa5cc5e565..4813641e1b7b 100644 --- a/drivers/cxl/core/suspend.c +++ b/drivers/cxl/core/suspend.c @@ -2,9 +2,14 @@ /* Copyright(c) 2022 Intel Corporation. All rights reserved. */ #include #include +#include #include "cxlmem.h" +#include "cxlpci.h" static atomic_t mem_active; +static atomic_t pci_loaded; + +static DECLARE_WAIT_QUEUE_HEAD(cxl_wait_queue); bool cxl_mem_active(void) { @@ -14,6 +19,7 @@ bool cxl_mem_active(void) void cxl_mem_active_inc(void) { atomic_inc(&mem_active); + wake_up(&cxl_wait_queue); } EXPORT_SYMBOL_NS_GPL(cxl_mem_active_inc, "CXL"); @@ -22,3 +28,38 @@ void cxl_mem_active_dec(void) atomic_dec(&mem_active); } EXPORT_SYMBOL_NS_GPL(cxl_mem_active_dec, "CXL"); + +void mark_cxl_pci_loaded(void) +{ + atomic_inc(&pci_loaded); + wake_up(&cxl_wait_queue); +} +EXPORT_SYMBOL_NS_GPL(mark_cxl_pci_loaded, "CXL"); + +static bool cxl_pci_loaded(void) +{ + if (IS_ENABLED(CONFIG_CXL_PCI)) + return atomic_read(&pci_loaded) != 0; + + return true; +} + +static bool cxl_mem_probed(void) +{ + if (IS_ENABLED(CONFIG_CXL_MEM)) + return atomic_read(&mem_active) != 0; + + return true; +} + +void cxl_wait_for_pci_mem(void) +{ + if (IS_ENABLED(CONFIG_CXL_PCI) || IS_ENABLED(CONFIG_CXL_MEM)) + if (wait_event_timeout(cxl_wait_queue, + cxl_pci_loaded() && cxl_mem_probed(), + 30 * HZ)) { + pr_debug("Timeout waiting for CXL PCI or CXL Memory probing"); + } + +} +EXPORT_SYMBOL_NS_GPL(cxl_wait_for_pci_mem, "CXL"); diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index be8a7dc77719..40835ec692c8 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -858,6 +858,7 @@ bool is_cxl_pmem_region(struct device *dev); struct cxl_pmem_region *to_cxl_pmem_region(struct device *dev); int cxl_add_to_region(struct cxl_port *root, struct cxl_endpoint_decoder *cxled); +int cxl_region_srmem_update(void); struct cxl_dax_region *to_cxl_dax_region(struct device *dev); u64 cxl_port_get_spa_cache_alias(struct cxl_port *endpoint, u64 spa); #else @@ -902,6 +903,8 @@ void cxl_coordinates_combine(struct access_coordinate *out, bool cxl_endpoint_decoder_reset_detected(struct cxl_port *port); +void cxl_wait_for_pci_mem(void); + /* * Unit test builds overrides this to __weak, find the 'strong' version * of these symbols in tools/testing/cxl/. diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 3ec6b906371b..1bd1e88c4cc0 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -853,17 +853,8 @@ int cxl_trigger_poison_list(struct cxl_memdev *cxlmd); int cxl_inject_poison(struct cxl_memdev *cxlmd, u64 dpa); int cxl_clear_poison(struct cxl_memdev *cxlmd, u64 dpa); -#ifdef CONFIG_CXL_SUSPEND void cxl_mem_active_inc(void); void cxl_mem_active_dec(void); -#else -static inline void cxl_mem_active_inc(void) -{ -} -static inline void cxl_mem_active_dec(void) -{ -} -#endif int cxl_mem_sanitize(struct cxl_memdev *cxlmd, u16 cmd); diff --git a/drivers/cxl/cxlpci.h b/drivers/cxl/cxlpci.h index 54e219b0049e..5a811ac63fcf 100644 --- a/drivers/cxl/cxlpci.h +++ b/drivers/cxl/cxlpci.h @@ -135,4 +135,5 @@ void read_cdat_data(struct cxl_port *port); void cxl_cor_error_detected(struct pci_dev *pdev); pci_ers_result_t cxl_error_detected(struct pci_dev *pdev, pci_channel_state_t state); +void mark_cxl_pci_loaded(void); #endif /* __CXL_PCI_H__ */ diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 4288f4814cc5..b784008489b3 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -1185,6 +1185,8 @@ static int __init cxl_pci_driver_init(void) if (rc) pci_unregister_driver(&cxl_pci_driver); + mark_cxl_pci_loaded(); + return rc; } diff --git a/include/linux/pm.h b/include/linux/pm.h index 78855d794342..11ff485c9722 100644 --- a/include/linux/pm.h +++ b/include/linux/pm.h @@ -35,14 +35,7 @@ static inline void pm_vt_switch_unregister(struct device *dev) } #endif /* CONFIG_VT_CONSOLE_SLEEP */ -#ifdef CONFIG_CXL_SUSPEND bool cxl_mem_active(void); -#else -static inline bool cxl_mem_active(void) -{ - return false; -} -#endif /* * Device power management From patchwork Thu Apr 3 18:33:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Terry Bowman X-Patchwork-Id: 878097 Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-sn1nam02on2078.outbound.protection.outlook.com [40.107.96.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 91A9B253333; Thu, 3 Apr 2025 18:34:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.96.78 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743705244; cv=fail; b=P6W0JKxFkOGQlNS1WnBPGr7FP8CHzNRYOkfjh4dpN1jzNRz8gZ3d77iC/MQHvjP3WULDKFJf2kROUVAu5gX2iauKZ+Snz6/gnAw2kn2wSV0xPRRcHmeXn+uiWSktGcmzQ40eDDbxEg4xda8TpbwMZVSzvcj1rdagDeR4j+GjuyU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743705244; c=relaxed/simple; bh=/qnry4eeyZdPgksHohXH1VNB8FWzcPSaUS9VmrJINew=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=LUoX07sGpmqQO6x650u4khe6IartTuuBuku8iOPwcBmSO4btao7Z07PR1XDhlTr2LhGN0WNeYLtEIuiungkvLBU2T3+6HD0xz9747vh1erOE9X72f7P32EtrfWbAFoZP06uPlR93K6hAfrzxavGmYN/UgLH6/wxWKLL3+xiQvEM= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=kRxFS2CU; arc=fail smtp.client-ip=40.107.96.78 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="kRxFS2CU" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=agMUnpk+1qIbP0xpuf8l4go1UC0Uqc56Ps5zmjTEWgL61JMLeSgoqXPK6Znwi7gm0SkQQ2RKbJ6tYl6HOrHNyPBwRWIw8aYuYs0hIddFpfa/Z0/GUfkLuGk0c1Rlbs7whi4ULeyqHKqzpu2NRxAIvl7LPLsKaIBYnSAzqkOMEKGmryic3OQHk+gL1bBdy//kbaextcbj7eAWhqCe0uunkrTcFtjSfyDLOCxkH+3hdmGw9x+53zMKl9MqUfnD9gokDOImdAkOFZ6UQe8pPFgqU3EgcOhDOYJ7JzZzfrQ3MG322Gw/LeSBnwbPHNv3+ZnieKNwxeLxL+9APYJd2eOtGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=D+8doWpGPDpi9+d+tVbEJhntGgQB8sRjpWP7OjYrTHQ=; b=oXE/+lfUZlq4DBz1uAuPrSOQWE+XS8fn/SwdqEPfcslV/cZp1ZwNmR5vgp27qXm/Tbwb4Dm0i820XxzYs2yAgMqZYIb/3uLTIuvqlVeiWjEFdmIf0p38K/YFE2SwKhv05hyfucGWeS1UTk8hUcEeBg/RP1/VaBBajzKlcyiGGFFUvj2qhOImAa0s2khgzfoVS1DBTvPQuZIIHipYkePSkzNkoHIPEHP3vEgvno4Hvc3+62sJmgXOwt/CHja0EdpjdkIpi1Q0I8qZwB21UazZ0CLlZ9z6rwBAwhH1AOmRAb1mJxV7dx29ujRgwHTzp6vUYGW8xUBPQKrdaAIA2HBsRw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=D+8doWpGPDpi9+d+tVbEJhntGgQB8sRjpWP7OjYrTHQ=; b=kRxFS2CUVyYcFvk5V03r3urx1j9zLhMhC+mzM0ik7pqcDMzZtdWZ+14U2frDQgbEOVXytL/Dd1kXF9MWRKJ1+xt7v8FXYTiTaw2QLD72rKPrZJhc1H3FeBrg5dKt6jvznGVW74nVKJuppdWJTMp7v2fPqtQbH8G8FlKJqerR5e0= Received: from MW4PR03CA0254.namprd03.prod.outlook.com (2603:10b6:303:b4::19) by SJ5PPF75EAF8F39.namprd12.prod.outlook.com (2603:10b6:a0f:fc02::999) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.45; Thu, 3 Apr 2025 18:33:58 +0000 Received: from MWH0EPF000971E2.namprd02.prod.outlook.com (2603:10b6:303:b4:cafe::2e) by MW4PR03CA0254.outlook.office365.com (2603:10b6:303:b4::19) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.44 via Frontend Transport; Thu, 3 Apr 2025 18:33:58 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MWH0EPF000971E2.mail.protection.outlook.com (10.167.243.69) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8606.22 via Frontend Transport; Thu, 3 Apr 2025 18:33:57 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Thu, 3 Apr 2025 13:33:56 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v3 3/4] dax/mum: Save the dax mum platform device pointer Date: Thu, 3 Apr 2025 13:33:14 -0500 Message-ID: <20250403183315.286710-4-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250403183315.286710-1-terry.bowman@amd.com> References: <20250403183315.286710-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000971E2:EE_|SJ5PPF75EAF8F39:EE_ X-MS-Office365-Filtering-Correlation-Id: 896ba358-8e4b-491a-5c92-08dd72de18c5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|376014|36860700013|7416014|82310400026|1800799024|921020; X-Microsoft-Antispam-Message-Info: mEGuDp4lSsqIuimAvASJX3zi7+l6tYEWFexduSQSGeFjGOnFvcdQb6w8GaSKb1bXdewdWHDgCGx7Zk/UuEafOoKt4f9MeBjrvo2nmx+IkQDZDCc8otzKR1m2YohbqgEOY5vEt/D5Fnuz3W5AhkKopvDtiGFKKb0ZMROJ4jcdWU99A1F21vT86UHPf9fNhMN9VV8M4tRH0vnIVNLIm4Be6/WhkzquXCipNFTqXqPXFqU7GHyOX0vXzr7D4rJjlJlgUpfY5KB2peMxWXzwSaTF9tdOtJNkuwVCg6/ZLfeWSW2L5oza30vNw3APMmQj+1upi6Dn4fgYYcJxAzNopnAh73qfRLtS0+LWFl8QsJNIAT53Cf/xAh09vRDA/Joa2VOnpH0kK/TkZrot80NY7nsAvdeF1TmzDfCmw+okCpyzvB373rFFhIaPGEatqpjOdt4dDb2HF2hvruugIQEDH/sYIbA4dea0UM0wRv1YSUaKZ9ZSLLpxqwEK3qXLw2zjYPcZNKPVm2l7HwVlYP7tuCTQCH5WaDowhllqw2J5gq0GiVwYV3rFLGTnMqQyAjDev5fQNjALwwk33XAedUVLu9zuN1mUvyOSGcW9jUyPmqd5S71OeT+KXt288zNKQbpfghIQUyuS6RXgV6MHB0ZG1+ajobhGwqTVgrZe69mfNfiWJns2couDh1nG4SNobWybrUsCH0Vz/d2kavYGfjKyMe2KaL1tu9BBdLPK7quy1iYrgswPknO1cB3lnuDV2BH7/BPD9dGG/lSEv4djnOlnztI+zjTamRBphzD1TFPDmr09Vih8T1nG6UY34hzEc7ZhZ0KNbyPtfcfQ9Jg3wAtzXrWPiQFJI0Z11nQFbdRiX+Jf6qaJ/GF27wIUvbuVXTp3JK5SkeVXXFobaOKfWpyL0XOq/URlyuU7oHI/cAlZhr3pPwP051XA8cNHeQdqF16PtYdiV9/GLCphyFgAsCTU0xpo0L3lKtpWngI5N+T+NmGzyKMqy1Me0J1XhNPTpNbPY5BLtlzj8NV1f7Ps/cJhls4UW4XccgYOb6lS77uGUNAklMynnaxqo7b674f05kQsOHUpEdvXwnrW0vOoMoQtlhog9pA3lvC2uk0LhW3Bqjuav2aiolzbM1kDFkujnfCs88MZe/QCLJea3Q6fBySV9+3gYEHfPPcdQxcR9sGQTv8EV8fPAMgjM06QY2ju9ACFYan0WE1zUb8iQVm6iqC1Bfm638ET1AgKo04KHzrvagW9dPgW2+fvBQkm95zqTjpmMgJBzPJR5HKNW2/2nS9KVBaDO5lrAUf2WXw6qfm+CBMK5GQuzxLkGNX6ePWYf3bTig4bvtvDKGR+OvMBQ6N8JfJmXY44jgvAdDjHikgXs2UbJtdH/9/YldxuAbGSx54trFP/wbPlp7qiJvfxYZV9ZxQDKUO3yImSQy9j+55Rfu2VWmw9QlG2VB2kTGbUnkT1wegX6pOVSppU6EqMrM2OPzd7JQ6XPBWPyf+LaiqD+dpfEzs= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(376014)(36860700013)(7416014)(82310400026)(1800799024)(921020); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2025 18:33:57.8965 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 896ba358-8e4b-491a-5c92-08dd72de18c5 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000971E2.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ5PPF75EAF8F39 From: Nathan Fontenot In order to handle registering hmem devices for SOFT RESERVE resources after the dax hmem device initialization occurs we need to save a reference to the dax hmem platform device that will be used in a following patch. Saving the platform device pointer also allows us to clean up the walk_hmem_resources() routine to no require the struct device argument. There should be no functional changes. Signed-off-by: Nathan Fontenot Signed-off-by: Terry Bowman --- drivers/dax/hmem/device.c | 4 ++-- drivers/dax/hmem/hmem.c | 9 ++++++--- include/linux/dax.h | 5 ++--- 3 files changed, 10 insertions(+), 8 deletions(-) diff --git a/drivers/dax/hmem/device.c b/drivers/dax/hmem/device.c index f9e1a76a04a9..59ad44761191 100644 --- a/drivers/dax/hmem/device.c +++ b/drivers/dax/hmem/device.c @@ -17,14 +17,14 @@ static struct resource hmem_active = { .flags = IORESOURCE_MEM, }; -int walk_hmem_resources(struct device *host, walk_hmem_fn fn) +int walk_hmem_resources(walk_hmem_fn fn) { struct resource *res; int rc = 0; mutex_lock(&hmem_resource_lock); for (res = hmem_active.child; res; res = res->sibling) { - rc = fn(host, (int) res->desc, res); + rc = fn((int) res->desc, res); if (rc) break; } diff --git a/drivers/dax/hmem/hmem.c b/drivers/dax/hmem/hmem.c index 5e7c53f18491..3aedef5f1be1 100644 --- a/drivers/dax/hmem/hmem.c +++ b/drivers/dax/hmem/hmem.c @@ -9,6 +9,8 @@ static bool region_idle; module_param_named(region_idle, region_idle, bool, 0644); +static struct platform_device *dax_hmem_pdev; + static int dax_hmem_probe(struct platform_device *pdev) { unsigned long flags = IORESOURCE_DAX_KMEM; @@ -59,9 +61,9 @@ static void release_hmem(void *pdev) platform_device_unregister(pdev); } -static int hmem_register_device(struct device *host, int target_nid, - const struct resource *res) +static int hmem_register_device(int target_nid, const struct resource *res) { + struct device *host = &dax_hmem_pdev->dev; struct platform_device *pdev; struct memregion_info info; long id; @@ -125,7 +127,8 @@ static int hmem_register_device(struct device *host, int target_nid, static int dax_hmem_platform_probe(struct platform_device *pdev) { - return walk_hmem_resources(&pdev->dev, hmem_register_device); + dax_hmem_pdev = pdev; + return walk_hmem_resources(hmem_register_device); } static struct platform_driver dax_hmem_platform_driver = { diff --git a/include/linux/dax.h b/include/linux/dax.h index df41a0017b31..4b4d16f94898 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -277,7 +277,6 @@ static inline void hmem_register_resource(int target_nid, struct resource *r) } #endif -typedef int (*walk_hmem_fn)(struct device *dev, int target_nid, - const struct resource *res); -int walk_hmem_resources(struct device *dev, walk_hmem_fn fn); +typedef int (*walk_hmem_fn)(int target_nid, const struct resource *res); +int walk_hmem_resources(walk_hmem_fn fn); #endif From patchwork Thu Apr 3 18:33:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Terry Bowman X-Patchwork-Id: 878540 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2065.outbound.protection.outlook.com [40.107.93.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B09E253354; Thu, 3 Apr 2025 18:34:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.93.65 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743705260; cv=fail; b=brEa7fFRLpB3D1DtlJlQoQLOvQXwj3fzdy+wu52sRWVECKZ9ntXpQwI3cfyyOIYLtSqlPhVTPoANCeCjwIXbt0tl9xMujrpgRXKFA8AWNd7hqzZeUyHAlz42jyxI4bgFCUlWuK6Q7CO84NJlQLvczGp4lJEgk/hnOnFSOYEOdmU= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743705260; c=relaxed/simple; bh=G3c0tHgzlvHJ/3FvYX3yOz67ej/rNF/u4xOVL6yUB64=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=T1B+SVZTSkmJfHbKFdZp8dAME3oJlH/QlRrtScTVnuECMfUkgUpbIXLucR7ObdAth7LrOv8whi9ysauhVLojwK9J3igEPrptig0IomIxLnBkUbO0lwUUpfl8BRxyU5sjzyMQLZnJmURkEu+FGFyj7ML6XQwPSu8UFDFmAikqhZ8= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com; spf=fail smtp.mailfrom=amd.com; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b=ZM9/aZMO; arc=fail smtp.client-ip=40.107.93.65 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=amd.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=amd.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=amd.com header.i=@amd.com header.b="ZM9/aZMO" ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=asmtTMVCxZWphFK+QE9VEoZCpPzN0lV0ICyW1xegSRM5lf4QdGekXPFSjZVbz3TeWDp0Rwk3NWoq5LWM1Fs34VFQT98gikdRE8e8s8mFJm23ZOzrWJKYeVOqIb+rnwNnWKPfXFkuR3zWv+SbJjeacHqY6l/Zf2CmjP0pPGiEm+wxzyhNJp0up7V7piDE4DF5v85oWT/AmqosyNuNeT3JbLHPHIROXOO0uYeZ/KjGzPDpTuRAMZ392M/NCdly1KZY5DdylRFklcNdzrkF7CPr18u/71b+aIAnjkGFAnmluDw9fM2pcADN4Fy/bW9Gw0zF+8YtWF05/kFuTjRhqNSbOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fY+xdnr5O6gTc+IhEpoy46Ls+W70O7ZGJaHQ964CO1g=; b=cTJeOO2p4U/nI/3wZr5tApPmRxa5Oob+kold4QdCZjdbJJYOv68Be4iEcYcagx404U6lIhJXTRllHkih8IV740WXsPC/tbM/xvrIbWvfdfL51y4Mxidl1fMtQVwtxinoSuCKmdz7eBh7jJ2x7CRuuBw9ZPxbKHzNkuKCHs3sDu0+gCxRwmWYZouQ/GAIrpQj4sK4YT6yHIP0a4NxYKSjyBxECUkKddHXn/RA/tx2uSMEp4r6zpVQTzgvulQRjsvbt+6KQaW0mTF7AAa8TkdNKWRcFDHA/mdhrx6V5JXuQCKHe8rYVBR9UDAo/uAwLYCvutounX8khTVlNjHdIxK5QQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=stgolabs.net smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fY+xdnr5O6gTc+IhEpoy46Ls+W70O7ZGJaHQ964CO1g=; b=ZM9/aZMOGPBHQ68ytRrOpS08LGy5tFzPiBHqm+Z+QQwexNuVv295PF7zeJiNVq+o9SEii+jHRljfc8b0iTCBBpbsIdQotpmmj6TY9qtAbAZjD9VB2dJSxWPmQxaCSx9UXvW4y04XQSFFA/RbWFzuQXegSpolgvbXlJPIYn7ZvyE= Received: from BYAPR07CA0006.namprd07.prod.outlook.com (2603:10b6:a02:bc::19) by DS0PR12MB8319.namprd12.prod.outlook.com (2603:10b6:8:f7::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.49; Thu, 3 Apr 2025 18:34:12 +0000 Received: from MWH0EPF000971E8.namprd02.prod.outlook.com (2603:10b6:a02:bc:cafe::6d) by BYAPR07CA0006.outlook.office365.com (2603:10b6:a02:bc::19) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8583.42 via Frontend Transport; Thu, 3 Apr 2025 18:34:09 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MWH0EPF000971E8.mail.protection.outlook.com (10.167.243.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8606.22 via Frontend Transport; Thu, 3 Apr 2025 18:34:09 +0000 Received: from ethanolx7ea3host.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Thu, 3 Apr 2025 13:34:07 -0500 From: Terry Bowman To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v3 4/4] cxl/dax: Delay consumption of SOFT RESERVE resources Date: Thu, 3 Apr 2025 13:33:15 -0500 Message-ID: <20250403183315.286710-5-terry.bowman@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250403183315.286710-1-terry.bowman@amd.com> References: <20250403183315.286710-1-terry.bowman@amd.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000971E8:EE_|DS0PR12MB8319:EE_ X-MS-Office365-Filtering-Correlation-Id: 4def0378-e14e-4595-d805-08dd72de1faa X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; ARA:13230040|7416014|1800799024|82310400026|376014|36860700013|921020; X-Microsoft-Antispam-Message-Info: CNvYzSJyrB2zRTgzSZ1ueUXea751EXAF2o5Uqzgj1hEPxMhea3K6T1Mqw6iw5D0J7rCMsWfz6F5Ay7bl5AN4Za9kg0gTIUzqNRLi1opxLyj42SbZ7Wya0MuUDhih591ud0aklSm9UEId6VtThpuwOYAFvl9YJNIylKy5htD+wXjFDj+4WS0Qk5vFImNr8RJWWtxHK5QCDDGLwB47cZ57ZIZwjdNCiy1uA5dPf/RE16KBtbCNeNS1iK3RL0cBnEe2Qlg1v3aWawBfgI9XwIsURGZaT+a4J3TRaPssPLmTbzKmueHqDEZGT1kPFrVkZaPBl8Na1rX2+2jYFnDwdqjn8Jy/vO9f5p4qVRS1M1BVTRe2cQ/BK7J9Ncv3/nevtPO85vWEAMM5rmWq6XHk8qoW6dL9akYZVxNIIEnKKoAH40Wy8Jfx+f3lyNk3RACapsKj/TyHG/YOeFKgwTwStcPoA8JsiHFAObcCt1inMDfXrFwEo7PxAh+tq7KuUFBZ5nLYJQwM0sTVXy3FYpBqxDAAQvccrvR/XkqLR6p5QcYu495zD5Lvl1jepa4YJprHmYaT6sCL772esky3hcq1yyLipg5CpIGORS7/OCeka6oiXRN3OhWRuGjgC0zmvq/6P212unkRyHb6flgjmzXRex7u/ydCo6xqWpdMnOdrTxm6WHQqpp9Tl6Ul75E52xDyGaJOr5PdtzvvVwG2GJb6US/P12EEFhuTTUcRJA8QRDZMmMPmZjglnec+48v5HWofKk9DSkqd0aA1qjirdkwPcJ1aMuQlI4DNYyOmC2xJN407BWXwZ94jHL4eNLADwda34o1X6n/vtxOihu7YOtTp+IMXKZiVGdN4a2jrnusuPD7tX4y/mjw9BZlYuzuvlkvxbm+CtdS4jDsySHFvKVNLSLayvRi8pQeu/glIFfnL+Y5t73wL0h9f4GfSqdLf0vdckANvMj+zSCcXiFs1af230o/N0xF6nLJqN7CJNAy1mRMxq2pjzSgLsYQaN89KulK8yic4pB/aAB2qhz7xIJoU0T5OwDQKw4C7ellmv5Ag3e5le6LrUBLu/QI/BbfUUKdD0IuFn6CZhqG41PBp0T+DuGIUZVAlrbd+Tp1YUcEYCn5oepJu6xcipzPIradVUpiJexsZf+5w2pdfl7eCC2spEsxU3dI8y/7TCAfgnwTiN7A+9zNfUCvCxCNIItrNUwqb2TRd0nZAlTb97mtViSp7Lvnm4scOCJcfnv4VbOAomS5V6H73WzRXOp7p4eHs3N7N08tbVIyDUn9pUG1PjfGonJJWHE3Dpc8HT194vPRx70zrguyIb+aLZWBFhKkZqqLbMrEN2UO2h9b6Lz6ud5nueTO3+pYW8LiN5ELOAxEBW7y0/a5s9qznhZxLC+UMNKrQvmyuxsYioeogtwvA3XwrLnQmj+utyHYSz5Rk0+svC0J/y6GJoDsGO+MmseTswLTJLoCqfucqY0lxy07YRBlrAiY3unPi3xzFVQX3fmi2CLGzuG4= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230040)(7416014)(1800799024)(82310400026)(376014)(36860700013)(921020); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2025 18:34:09.4512 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4def0378-e14e-4595-d805-08dd72de1faa X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000971E8.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8319 From: Nathan Fontenot The dax hmem device initialization will consume any iomem SOFT RESERVE resources prior to CXL region creation. To allow for the CXL driver to complete region creation and trim any SOFT RESERVE resources before the dax driver consumes them we need to delay the dax driver's search for SOFT RESERVEs. To do this the dax driver hmem device initialization code skips the walk of the iomem resource tree if the CXL ACPI driver is enabled. This allows the CXL driver to complete region creation and trim any SOFT RESERVES. Once the CXL driver completes this, the CXL driver then registers any remaining SOFT RESERVE resources with the dax hmem driver. Signed-off-by: Nathan Fontenot Signed-off-by: Terry Bowman --- drivers/cxl/core/region.c | 10 +++++++++ drivers/dax/hmem/device.c | 43 ++++++++++++++++++++------------------- drivers/dax/hmem/hmem.c | 3 ++- include/linux/dax.h | 6 ++++++ 4 files changed, 40 insertions(+), 22 deletions(-) diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c index 25d70175f204..bf4a4371d98b 100644 --- a/drivers/cxl/core/region.c +++ b/drivers/cxl/core/region.c @@ -11,6 +11,7 @@ #include #include #include +#include #include #include #include "core.h" @@ -3444,6 +3445,11 @@ int cxl_add_to_region(struct cxl_port *root, struct cxl_endpoint_decoder *cxled) } EXPORT_SYMBOL_NS_GPL(cxl_add_to_region, "CXL"); +static int cxl_srmem_register(struct resource *res, void *unused) +{ + return hmem_register_device(phys_to_target_node(res->start), res); +} + int cxl_region_srmem_update(void) { struct device *dev = NULL; @@ -3461,6 +3467,10 @@ int cxl_region_srmem_update(void) put_device(dev); } while (dev); + /* Now register any remaining SOFT RESERVES with dax */ + walk_iomem_res_desc(IORES_DESC_SOFT_RESERVED, IORESOURCE_MEM, + 0, -1, NULL, cxl_srmem_register); + return 0; } EXPORT_SYMBOL_NS_GPL(cxl_region_srmem_update, "CXL"); diff --git a/drivers/dax/hmem/device.c b/drivers/dax/hmem/device.c index 59ad44761191..cc1ed7bbdb1a 100644 --- a/drivers/dax/hmem/device.c +++ b/drivers/dax/hmem/device.c @@ -8,7 +8,6 @@ static bool nohmem; module_param_named(disable, nohmem, bool, 0444); -static bool platform_initialized; static DEFINE_MUTEX(hmem_resource_lock); static struct resource hmem_active = { .name = "HMEM devices", @@ -35,9 +34,7 @@ EXPORT_SYMBOL_GPL(walk_hmem_resources); static void __hmem_register_resource(int target_nid, struct resource *res) { - struct platform_device *pdev; struct resource *new; - int rc; new = __request_region(&hmem_active, res->start, resource_size(res), "", 0); @@ -47,21 +44,6 @@ static void __hmem_register_resource(int target_nid, struct resource *res) } new->desc = target_nid; - - if (platform_initialized) - return; - - pdev = platform_device_alloc("hmem_platform", 0); - if (!pdev) { - pr_err_once("failed to register device-dax hmem_platform device\n"); - return; - } - - rc = platform_device_add(pdev); - if (rc) - platform_device_put(pdev); - else - platform_initialized = true; } void hmem_register_resource(int target_nid, struct resource *res) @@ -83,9 +65,28 @@ static __init int hmem_register_one(struct resource *res, void *data) static __init int hmem_init(void) { - walk_iomem_res_desc(IORES_DESC_SOFT_RESERVED, - IORESOURCE_MEM, 0, -1, NULL, hmem_register_one); - return 0; + struct platform_device *pdev; + int rc; + + if (!IS_ENABLED(CONFIG_CXL_ACPI)) { + walk_iomem_res_desc(IORES_DESC_SOFT_RESERVED, + IORESOURCE_MEM, 0, -1, NULL, + hmem_register_one); + } + + pdev = platform_device_alloc("hmem_platform", 0); + if (!pdev) { + pr_err("failed to register device-dax hmem_platform device\n"); + return -1; + } + + rc = platform_device_add(pdev); + if (rc) { + pr_err("failed to add device-dax hmem_platform device\n"); + platform_device_put(pdev); + } + + return rc; } /* diff --git a/drivers/dax/hmem/hmem.c b/drivers/dax/hmem/hmem.c index 3aedef5f1be1..a206b9b383e4 100644 --- a/drivers/dax/hmem/hmem.c +++ b/drivers/dax/hmem/hmem.c @@ -61,7 +61,7 @@ static void release_hmem(void *pdev) platform_device_unregister(pdev); } -static int hmem_register_device(int target_nid, const struct resource *res) +int hmem_register_device(int target_nid, const struct resource *res) { struct device *host = &dax_hmem_pdev->dev; struct platform_device *pdev; @@ -124,6 +124,7 @@ static int hmem_register_device(int target_nid, const struct resource *res) platform_device_put(pdev); return rc; } +EXPORT_SYMBOL_GPL(hmem_register_device); static int dax_hmem_platform_probe(struct platform_device *pdev) { diff --git a/include/linux/dax.h b/include/linux/dax.h index 4b4d16f94898..a1a75ade9ea7 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -271,10 +271,16 @@ static inline int dax_mem2blk_err(int err) #ifdef CONFIG_DEV_DAX_HMEM_DEVICES void hmem_register_resource(int target_nid, struct resource *r); +int hmem_register_device(int target_nid, const struct resource *res); #else static inline void hmem_register_resource(int target_nid, struct resource *r) { } + +static inline int hmem_register_device(int target_nid, const struct resource *res) +{ + return 0; +} #endif typedef int (*walk_hmem_fn)(int target_nid, const struct resource *res);