From patchwork Wed May 14 19:34:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mario Limonciello X-Patchwork-Id: 890816 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 717CB2920AD; Wed, 14 May 2025 19:34:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747251277; cv=none; b=hmSDbbyV8eKMJbYjG9eEbw0RU2qek4tYUaqRmywMVmvBDzIPjXEVPl1kfXuMMFwA5724TyfyeVfHHH2w1Zdf8xvB3/JAEzlmXIqZyKi9cKrUoDDdWQz6UvZa5iBbaLIDmD2y/V3DUULPUfTkQ2vxOGyAiSZJ5LX7v4KPt5WTKk8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747251277; c=relaxed/simple; bh=3ivflG2b+7xFcnqYwqOf1n0ZgZzhrPJqm9pltBGm89A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=esHnM0I5AldKTURkB/U2vePn7oRpqCsRjYGPQiCnyiroGrw9SijCp+6h0V9Gd3fzAC3hPjhJy/GB2+uCsnzy7fWduFyeO8A2Xazgc5fsaVhkPh55Yz/roKVhp4B4fmTYaNvTVmFXi8IY39g1rEJysXa+IFg5bT7u7TpLgQ3z0Vc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DXUYXXhh; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DXUYXXhh" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BE84FC4CEF0; Wed, 14 May 2025 19:34:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1747251276; bh=3ivflG2b+7xFcnqYwqOf1n0ZgZzhrPJqm9pltBGm89A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DXUYXXhh1Lp6DWU4WJjVQppnWAxqGKSCRSpGi3CJfEniKt3svhcIM9CX3bR3s0x6F SD/fmcO36MpRMYWCIpqiQdDirGY4TgI8yVhVJWe0XvAOtP6A57D2EqhgmwutMrG2gz q91V3jgyJLR4B9XT98Gwlx6MlNO6tg+e3Miu1QtR3laCfTo38PKzT4hLIbmW+Eww/3 1HP0zI14cnhdOoJLjEUhtf6cuCOHuZ7bpOXpR0SZSUwozjqzTa8veODfabMRtvg4wD kx2Ul/DL0X7R1C8nMbXzzgs2fICc61tZGNKF5jzzG3UajOqwAqC32+bj1baAUy8sCB 7ZLR5Vc4jOReg== From: Mario Limonciello To: "Rafael J . Wysocki" , Alex Deucher , Bjorn Helgaas Cc: amd-gfx@lists.freedesktop.org (open list:RADEON and AMDGPU DRM DRIVERS), linux-pm@vger.kernel.org (open list:HIBERNATION (aka Software Suspend, aka swsusp)), linux-pci@vger.kernel.org (open list:PCI SUBSYSTEM), linux-kernel@vger.kernel.org (open list), Mario Limonciello , AceLan Kao , Kai-Heng Feng , Mark Pearson , =?utf-8?q?Merthan_Karaka=C5=9F?= , Denis Benato Subject: [PATCH v2 1/3] PM: Use hibernate flows for system power off Date: Wed, 14 May 2025 14:34:04 -0500 Message-ID: <20250514193406.3998101-2-superm1@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250514193406.3998101-1-superm1@kernel.org> References: <20250514193406.3998101-1-superm1@kernel.org> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Mario Limonciello When the system is powered off the kernel will call device_shutdown() which will issue callbacks into PCI core to wake up a device and call it's shutdown() callback. This will leave devices in ACPI D0 which can cause some devices to misbehave with spurious wakeups and also leave some devices on which will consume power needlessly. The issue won't happen if the device is in D3 before system shutdown, so putting device to low power state before shutdown solves the issue. ACPI Spec 6.5, "7.4.2.5 System \_S4 State" says "Devices states are compatible with the current Power Resource states. In other words, all devices are in the D3 state when the system state is S4." The following "7.4.2.6 System \_S5 State (Soft Off)" states "The S5 state is similar to the S4 state except that OSPM does not save any context." so it's safe to assume devices should be at D3 for S5. To accomplish this, modify the PM core to call all the device hibernate callbacks when turning off the system when the kernel is compiled with hibernate support. If compiled without hibernate support or hibernate fails fall back into the previous shutdown flow. Cc: AceLan Kao Cc: Kai-Heng Feng Cc: Mark Pearson Cc: Merthan Karakaş Tested-by: Denis Benato Link: https://lore.kernel.org/linux-pci/20231213182656.6165-1-mario.limonciello@amd.com/ Link: https://lore.kernel.org/linux-pci/20250506041934.1409302-1-superm1@kernel.org/ Signed-off-by: Mario Limonciello --- v2: * Handle failures to hibernate (fall back to shutdown) * Don't use dedicated events * Only allow under CONFIG_HIBERNATE_CALLBACKS --- kernel/reboot.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/kernel/reboot.c b/kernel/reboot.c index ec087827c85cd..52f5e6e36a6f8 100644 --- a/kernel/reboot.c +++ b/kernel/reboot.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include #include @@ -305,6 +306,17 @@ static void kernel_shutdown_prepare(enum system_states state) (state == SYSTEM_HALT) ? SYS_HALT : SYS_POWER_OFF, NULL); system_state = state; usermodehelper_disable(); +#ifdef CONFIG_HIBERNATE_CALLBACKS + if (dpm_suspend_start(PMSG_HIBERNATE)) + goto resume_devices; + if (dpm_suspend_end(PMSG_HIBERNATE)) + goto resume_devices; + return; + +resume_devices: + pr_emerg("Failed to power off devices, using shutdown instead.\n"); + dpm_resume_end(PMSG_RESTORE); +#endif device_shutdown(); } /** From patchwork Wed May 14 19:34:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mario Limonciello X-Patchwork-Id: 890095 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E033293464; Wed, 14 May 2025 19:34:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747251278; cv=none; b=Fqc6YE/flI8/p3+4peAqJTaMVkUrPahfAJ6H0Jj54v7xsps8vP2VWfC28dIK6Vk4dqM8xB8EdLuQqrK1Ysvx8QpBP9yHS8cF8eTPCagSca3J9FVI2R4DroFNYKQ3F2Ep3yhE+MXWjiewKGUoElF38dgSmuppcdR8i0S6wT3EMCU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747251278; c=relaxed/simple; bh=s4QaK/a9MnnNl0w/we4occOHkkx2Oi4zoKvN8gfZ7pg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=Itrch50d4BmXsdeff6NpidT02udzO+qV2mc7iMKmhZlH4DCRYpxPQq87SrxmRBdId7SGMy8JntOICcBV1zkUH/wIiscORPtIMEfS2+ASpR9GhpUyg4D/4Qz1AuHRAEQAk/PHBdU8bSFSWUUXawd2AY5hLBNdS7fzEZZoXA/R4Zo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=alXMp7sj; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="alXMp7sj" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 301E1C4CEED; Wed, 14 May 2025 19:34:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1747251278; bh=s4QaK/a9MnnNl0w/we4occOHkkx2Oi4zoKvN8gfZ7pg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=alXMp7sjE7jtJNChcR7w0/J10J2dexexcJwdBkXbO8zOe9mqBS/UpDV6BRltWbTSf gOkFNU7urAUVSuWgRz4mtfVa2Ps4qBYp8I2O3NibdgjgOSmyEzxVNLmNsVFLfcQHAo krN/SQ5saVEX411pd1Oxn57U+Sr8TPREkN+2g9DikGWpanYIi7/Rj8bsCZ/f/STzLE 5salBgqy4CntfsN2QbGVTo94sh2XSzCjJHspdYgMhWzqI31hlLVus/wRhZj4JahNMa 67WPaqRUhXu3GQgMZ1vaA230caRH3CEy1pwkG5i1g72aolkv4CHTn2ud1vfH/Px4sA iRMxK/wvf7EYQ== From: Mario Limonciello To: "Rafael J . Wysocki" , Alex Deucher , Bjorn Helgaas Cc: amd-gfx@lists.freedesktop.org (open list:RADEON and AMDGPU DRM DRIVERS), linux-pm@vger.kernel.org (open list:HIBERNATION (aka Software Suspend, aka swsusp)), linux-pci@vger.kernel.org (open list:PCI SUBSYSTEM), linux-kernel@vger.kernel.org (open list), Mario Limonciello , AceLan Kao , Kai-Heng Feng , Mark Pearson , Denis Benato , =?utf-8?q?Merthan_Karaka=C5=9F?= Subject: [PATCH v2 2/3] PCI: Put PCIe ports with downstream devices into D3 at hibernate Date: Wed, 14 May 2025 14:34:05 -0500 Message-ID: <20250514193406.3998101-3-superm1@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250514193406.3998101-1-superm1@kernel.org> References: <20250514193406.3998101-1-superm1@kernel.org> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Mario Limonciello For the suspend flow PCIe ports that have downstream devices are put into the appropriate D3 state when children are not in D0. For the hibernate flow, PCIe ports with downstream devices stay in D0 however. This can lead to PCIe ports that are remained powered on needlessly during hibernate. Adjust the pci_pm_poweroff_noirq() to follow the same flow as pci_pm_suspend_noirq() in that PCIe ports that are power manageable should without downstream devices in D0 should be put into their appropriate sleep state. Cc: AceLan Kao Cc: Kai-Heng Feng Cc: Mark Pearson Cc: Denis Benato Cc: Merthan Karakaş Signed-off-by: Mario Limonciello --- drivers/pci/pci-driver.c | 39 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/drivers/pci/pci-driver.c b/drivers/pci/pci-driver.c index 0c5bdb8c2c07b..57eb129d57244 100644 --- a/drivers/pci/pci-driver.c +++ b/drivers/pci/pci-driver.c @@ -1141,6 +1141,8 @@ static int pci_pm_poweroff(struct device *dev) struct pci_dev *pci_dev = to_pci_dev(dev); const struct dev_pm_ops *pm = dev->driver ? dev->driver->pm : NULL; + pci_dev->skip_bus_pm = false; + if (pci_has_legacy_pm_support(pci_dev)) return pci_legacy_suspend(dev, PMSG_HIBERNATE); @@ -1204,8 +1206,35 @@ static int pci_pm_poweroff_noirq(struct device *dev) return error; } - if (!pci_dev->state_saved && !pci_has_subordinate(pci_dev)) - pci_prepare_to_sleep(pci_dev); + if (!pci_dev->state_saved) { + pci_save_state(pci_dev); + + /* + * If the device is a bridge with a child in D0 below it, + * it needs to stay in D0, so check skip_bus_pm to avoid + * putting it into a low-power state in that case. + */ + if (!pci_dev->skip_bus_pm && pci_power_manageable(pci_dev)) + pci_prepare_to_sleep(pci_dev); + } + + if (pci_dev->current_state == PCI_D0) { + pci_dev->skip_bus_pm = true; + /* + * Per PCI PM r1.2, table 6-1, a bridge must be in D0 if any + * downstream device is in D0, so avoid changing the power state + * of the parent bridge by setting the skip_bus_pm flag for it. + */ + if (pci_dev->bus->self) + pci_dev->bus->self->skip_bus_pm = true; + } + + if (pci_dev->skip_bus_pm && pm_suspend_no_platform()) { + pci_dbg(pci_dev, "PCI PM: Skipped\n"); + goto Fixup; + } + + pci_pm_set_unknown_state(pci_dev); /* * The reason for doing this here is the same as for the analogous code @@ -1214,6 +1243,7 @@ static int pci_pm_poweroff_noirq(struct device *dev) if (pci_dev->class == PCI_CLASS_SERIAL_USB_EHCI) pci_write_config_word(pci_dev, PCI_COMMAND, 0); +Fixup: pci_fixup_device(pci_fixup_suspend_late, pci_dev); return 0; @@ -1223,10 +1253,15 @@ static int pci_pm_restore_noirq(struct device *dev) { struct pci_dev *pci_dev = to_pci_dev(dev); const struct dev_pm_ops *pm = dev->driver ? dev->driver->pm : NULL; + pci_power_t prev_state = pci_dev->current_state; + bool skip_bus_pm = pci_dev->skip_bus_pm; pci_pm_default_resume_early(pci_dev); pci_fixup_device(pci_fixup_resume_early, pci_dev); + if (!skip_bus_pm && prev_state == PCI_D3cold) + pci_pm_bridge_power_up_actions(pci_dev); + if (pci_has_legacy_pm_support(pci_dev)) return 0; From patchwork Wed May 14 19:34:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Mario Limonciello X-Patchwork-Id: 890815 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 595E22949F9; Wed, 14 May 2025 19:34:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747251280; cv=none; b=a5CxwRIreNp67D2upvPUOWqdfgE9o1pM7rGObfwNsFNYCRK9N2SLQl/ECOnxx8bn24H6niqiTvfvMuwyn3GklezOuqBsakjHLdWJEfa6/xxPqzeYNqoSpz0tFXWzlsuFIwqymtTAa7GOS4LRWADiR29qFAcQm3Cy/uM9DJpc8ao= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747251280; c=relaxed/simple; bh=LB2ONc/3BqVLy79e00jHCVU/mzynv4kwHq33TPW/jAA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=tvXo/r4B7JFqTTZDcL953bjtfKf/wJsknv3d35FycLJjCjlcCnOR/dhKP6w1h+kGiCT9GJmDmcuDPAlTsnPHeBbABQDPR5fguTNgLyKAtdMrg4TJszAMrS82/siTTarSvKPE3Q8WXE9TV+kALrFoHOdExqybWad2hjE41pgQrqw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=DO6HimHt; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="DO6HimHt" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9D0D4C4CEF3; Wed, 14 May 2025 19:34:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1747251279; bh=LB2ONc/3BqVLy79e00jHCVU/mzynv4kwHq33TPW/jAA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DO6HimHtI31Dcm16iw6Zla/z/RpuGxcX0zMJ6uEjmvrljF1ktSnKgCKJfwbtYt5Bb lmyc0XfzMriIo2rHrUPGLPKiaSB4Kyk0Nz/+flbXZWKex+vFCsmYw6EQ/fy9eD8M1A BiXlGkm6AqsHymRfbyFMgreVS3YaGjUiQotn4ysD1di7Gz1VfBOHo0m+3BnBrum6/h jm5cHDNOz+VOUAVzVzEvKScbd3MH2pfD8rXOKIBXEnADMcm5sZx5927dimAuHrp1hx tNbGKPoMQt8gInI2K1B/9xFGlGjsr9voeciB+fSDsaaQHfWc45ZmoBIeuMWnziHaic cX2qCWAJ8XVuQ== From: Mario Limonciello To: "Rafael J . Wysocki" , Alex Deucher , Bjorn Helgaas Cc: amd-gfx@lists.freedesktop.org (open list:RADEON and AMDGPU DRM DRIVERS), linux-pm@vger.kernel.org (open list:HIBERNATION (aka Software Suspend, aka swsusp)), linux-pci@vger.kernel.org (open list:PCI SUBSYSTEM), linux-kernel@vger.kernel.org (open list), Mario Limonciello , AceLan Kao , Kai-Heng Feng , Mark Pearson , Denis Benato , =?utf-8?q?Merthan_Karaka=C5=9F?= Subject: [PATCH v2 3/3] drm/amd: Avoid evicting resources at S5 Date: Wed, 14 May 2025 14:34:06 -0500 Message-ID: <20250514193406.3998101-4-superm1@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250514193406.3998101-1-superm1@kernel.org> References: <20250514193406.3998101-1-superm1@kernel.org> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Mario Limonciello Normally resources are evicted on dGPUs at suspend or hibernate and on APUs at hibernate. These steps are unnecessary when using the S4 callbacks to put the system into S5. Cc: AceLan Kao Cc: Kai-Heng Feng Cc: Mark Pearson Cc: Denis Benato Cc: Merthan Karakaş Signed-off-by: Mario Limonciello --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 4d1b54f58495a..ea1385b6d894f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -4960,6 +4960,10 @@ static int amdgpu_device_evict_resources(struct amdgpu_device *adev) if (!adev->in_s4 && (adev->flags & AMD_IS_APU)) return 0; + /* No need to evict when going to S5 through S4 callbacks */ + if (system_state == SYSTEM_HALT || system_state == SYSTEM_POWER_OFF) + return 0; + ret = amdgpu_ttm_evict_resources(adev, TTM_PL_VRAM); if (ret) DRM_WARN("evicting device resources failed\n");