From patchwork Thu Jun 6 09:00:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Loehle X-Patchwork-Id: 802228 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 7D0F713D2A0; Thu, 6 Jun 2024 09:01:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717664478; cv=none; b=cHC0fo7mqU9czQcbTTWj/zd2VPPeDwGyhzPriKTwXKfI4kjheISjSECp+xs57EdJWKaLLaQfpSaKcbo+bKnJ2MJwbp6SR4XX7xkU7gh1CxM8GVPRJqDFRt8ChR1cY9pRZwsjPWt2pH8Ddni7zIEAHooXAGJQWYMSbJnhm3YubPc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717664478; c=relaxed/simple; bh=gPB/hPlo8TMgrOoFRBHzJovmHAJH3zJeXXoyccgpjsk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=E0pRyPRTPCSbtEk5w88fX/InWjhK/ecHWed8ICe55RzCpmng0UQJ223lAtURh9lb4/Ht9/5JwD/akRnMf2EO480jN3edRvOWz9fcgdi63knWvEit0B6h9+slY6bx7QQ3/PWP1C0AnLNhhEH7r0KFqrH9EhGbhwF/CeHAP8XC6ps= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 602CC339; Thu, 6 Jun 2024 02:01:40 -0700 (PDT) Received: from e127648.arm.com (unknown [10.57.71.137]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 3D6D73F762; Thu, 6 Jun 2024 02:01:13 -0700 (PDT) From: Christian Loehle To: linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, rafael@kernel.org Cc: vincent.guittot@linaro.org, qyousef@layalina.io, peterz@infradead.org, daniel.lezcano@linaro.org, anna-maria@linutronix.de, kajetan.puchalski@arm.com, lukasz.luba@arm.com, dietmar.eggemann@arm.com, Christian Loehle Subject: [PATCH 1/6] cpuidle: teo: Increase util-threshold Date: Thu, 6 Jun 2024 10:00:45 +0100 Message-Id: <20240606090050.327614-2-christian.loehle@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240606090050.327614-1-christian.loehle@arm.com> References: <20240606090050.327614-1-christian.loehle@arm.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Increase the util-threshold by a lot as it was low enough for some minor load to always be active, especially on smaller CPUs. For small cap CPUs (Pixel6) the util threshold is as low as 1. For CPUs of capacity <64 it is 0. So ensure it is at a minimum, too. Fixes: 9ce0f7c4bc64 ("cpuidle: teo: Introduce util-awareness") Reported-by: Qais Yousef Reported-by: Vincent Guittot Suggested-by: Kajetan Puchalski Signed-off-by: Christian Loehle --- drivers/cpuidle/governors/teo.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/drivers/cpuidle/governors/teo.c b/drivers/cpuidle/governors/teo.c index 7244f71c59c5..45f43e2ee02d 100644 --- a/drivers/cpuidle/governors/teo.c +++ b/drivers/cpuidle/governors/teo.c @@ -146,13 +146,11 @@ * The number of bits to shift the CPU's capacity by in order to determine * the utilized threshold. * - * 6 was chosen based on testing as the number that achieved the best balance - * of power and performance on average. - * * The resulting threshold is high enough to not be triggered by background - * noise and low enough to react quickly when activity starts to ramp up. + * noise. */ -#define UTIL_THRESHOLD_SHIFT 6 +#define UTIL_THRESHOLD_SHIFT 2 +#define UTIL_THRESHOLD_MIN 50 /* * The PULSE value is added to metrics when they grow and the DECAY_SHIFT value @@ -671,7 +669,8 @@ static int teo_enable_device(struct cpuidle_driver *drv, int i; memset(cpu_data, 0, sizeof(*cpu_data)); - cpu_data->util_threshold = max_capacity >> UTIL_THRESHOLD_SHIFT; + cpu_data->util_threshold = max(UTIL_THRESHOLD_MIN, + max_capacity >> UTIL_THRESHOLD_SHIFT); for (i = 0; i < NR_RECENT; i++) cpu_data->recent_idx[i] = -1; From patchwork Thu Jun 6 09:00:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Loehle X-Patchwork-Id: 802532 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 01BBE13AA5E; Thu, 6 Jun 2024 09:01:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717664482; cv=none; b=Qxo9GDjJZEpYlPfJ6m6QQNQkaZtllGpO4IASmOxoFrWpnCBeND7Mn7YRspBTirSFYsi6cS+ge2y0O4Kq21EQGf7WBozbPVaB/LrxmDOcAh3fUr//rRCmxxEX/kY6lHv863XI7b6Etdxovc+1La4d9kUsFm7W/e0YlmQgw83ZuNM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717664482; c=relaxed/simple; bh=qML0uzCbmFJloLReDcFy0oJJlPIuvZmfqSV9p2czb8M=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qJuRhzjFIRC7I4UuW0jsA9aYVPonoC42fU8BylagLFzl+NzBd5C5yGxm+ClZ+W0gtxItmSCtWbTBs3+cIxMRW55yCcuf1snQ/c8h61bAy3+Ci5gGoIjsCHrg5vm6j5LYZFBdfQPCXft68YN4WhIc67zGce2P/AOXNkmx4td0nY0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id DBC38339; Thu, 6 Jun 2024 02:01:44 -0700 (PDT) Received: from e127648.arm.com (unknown [10.57.71.137]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id E9E093F762; Thu, 6 Jun 2024 02:01:17 -0700 (PDT) From: Christian Loehle To: linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, rafael@kernel.org Cc: vincent.guittot@linaro.org, qyousef@layalina.io, peterz@infradead.org, daniel.lezcano@linaro.org, anna-maria@linutronix.de, kajetan.puchalski@arm.com, lukasz.luba@arm.com, dietmar.eggemann@arm.com, Christian Loehle Subject: [PATCH 2/6] cpuidle: teo: Don't stop tick on utilized Date: Thu, 6 Jun 2024 10:00:46 +0100 Message-Id: <20240606090050.327614-3-christian.loehle@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240606090050.327614-1-christian.loehle@arm.com> References: <20240606090050.327614-1-christian.loehle@arm.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 As we expect to be woken up early, stopping the tick is likely to be a waste. Signed-off-by: Christian Loehle --- drivers/cpuidle/governors/teo.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/cpuidle/governors/teo.c b/drivers/cpuidle/governors/teo.c index 45f43e2ee02d..2c427dd4cac0 100644 --- a/drivers/cpuidle/governors/teo.c +++ b/drivers/cpuidle/governors/teo.c @@ -429,10 +429,13 @@ static int teo_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, idx = 0; goto out_tick; } - /* Assume that state 1 is not a polling one and use it. */ + /* + * Assume that state 1 is not a polling one and use it, but + * don't stop the tick as we expect to be woken up early. + */ idx = 1; duration_ns = drv->states[1].target_residency_ns; - goto end; + goto out_tick_state; } /* Compute the sums of metrics for early wakeup pattern detection. */ @@ -618,6 +621,7 @@ static int teo_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, duration_ns >= TICK_NSEC) || tick_nohz_tick_stopped()) return idx; +out_tick_state: /* * The tick is not going to be stopped, so if the target residency of * the state to be returned is not within the time till the closest From patchwork Thu Jun 6 09:00:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Loehle X-Patchwork-Id: 802227 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id E604A14EC71; Thu, 6 Jun 2024 09:01:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717664487; cv=none; b=WLkiBQ/yLfm4AIuHwSVOtY2s/J7FpZG++YpKzpr2aLSwIFq8mBQg1dVTf6l7yLio+Ha3OVe4CR9//gsgsACZuvmUpJe2wHvzQfVS6TcGc7xnNgvQ5kidAe0Npag5Zf2aJkRtc9/Y4Wg3m7H4PSBUTV4THYRnhCfFeZXTJhOErns= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717664487; c=relaxed/simple; bh=Iy/Jw+bQQcXAeMSukRlrF4oA2JnT/oc9lz2u483pIYs=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=uBlYPvVBfXR91nvRVRLUEhX1EKVzBLx0IzBz6ZiBa+fLi+aDBsp1GYfrMXcKKYCM6n5gq916H91FRdTftSQOQM/RvTK/5TT6Th82elyodBxTa2xs5aMMy6Rq+RB2Zih+5kTQ4DfaUDd2COMAu2sEyCkkIAoi60n85XTu7kwCyl8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A6AC0339; Thu, 6 Jun 2024 02:01:49 -0700 (PDT) Received: from e127648.arm.com (unknown [10.57.71.137]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 8B4023F762; Thu, 6 Jun 2024 02:01:22 -0700 (PDT) From: Christian Loehle To: linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, rafael@kernel.org Cc: vincent.guittot@linaro.org, qyousef@layalina.io, peterz@infradead.org, daniel.lezcano@linaro.org, anna-maria@linutronix.de, kajetan.puchalski@arm.com, lukasz.luba@arm.com, dietmar.eggemann@arm.com, Christian Loehle Subject: [PATCH 3/6] cpuidle: teo: Don't always stop tick on one state Date: Thu, 6 Jun 2024 10:00:47 +0100 Message-Id: <20240606090050.327614-4-christian.loehle@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240606090050.327614-1-christian.loehle@arm.com> References: <20240606090050.327614-1-christian.loehle@arm.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Even if we have only one state, we unfortunately still need to query expected sleep length if state0 is a proper sleep state like WFI. This enables teo to save energy even in that scenario. Signed-off-by: Christian Loehle --- drivers/cpuidle/governors/teo.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) -- 2.34.1 diff --git a/drivers/cpuidle/governors/teo.c b/drivers/cpuidle/governors/teo.c index 2c427dd4cac0..216d34747e3b 100644 --- a/drivers/cpuidle/governors/teo.c +++ b/drivers/cpuidle/governors/teo.c @@ -403,7 +403,14 @@ static int teo_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, /* Check if there is any choice in the first place. */ if (drv->state_count < 2) { idx = 0; - goto out_tick; + if (drv->states[0].flags & CPUIDLE_FLAG_POLLING) + goto out_tick; + /* + * If we only have one state and it is a proper one, check if + * disabling the tick would be worth it. + */ + duration_ns = tick_nohz_get_sleep_length(&delta_tick); + goto end; } if (!dev->states_usage[0].disable) From patchwork Thu Jun 6 09:00:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Loehle X-Patchwork-Id: 802531 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 9D01914EC71; Thu, 6 Jun 2024 09:01:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717664490; cv=none; b=CNKAGfukCAH8Pc9ag0cpRgKuHhGisTzWhi10FUgEs5IZ0Fphgek0d8tWHMVaY4shryjYHFYOMAlcL9vKHzMR9y6wsspX6d62DFNu0ZxNO9rR3aDtADxs+rC3e9q6AGrJA0naFUf1cP2UdjS2OW5nCDGBe7GMhj+7gflgLWpm3UU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717664490; c=relaxed/simple; bh=BK15k8asB6S9tes/VSsFvDkpmsX7mPNMWzbLiswymSA=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EbZi35UqR9ylr3LTW+dp2h49oWZfu9XS/Imgk5uRkOkUgIWZlCAViNX8UgA9Dd44NWMpqqlf0X3IP1cF+CUe76PO6zK84frh65HWaCQZCQEecfi9jcFEKSwBRy0XheNPBsfLDQ0Th39971ycbaP0QSv6W1nsc8M0qSlPEzInbsw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 7A8F9339; Thu, 6 Jun 2024 02:01:53 -0700 (PDT) Received: from e127648.arm.com (unknown [10.57.71.137]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id ED7E63F762; Thu, 6 Jun 2024 02:01:26 -0700 (PDT) From: Christian Loehle To: linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, rafael@kernel.org Cc: vincent.guittot@linaro.org, qyousef@layalina.io, peterz@infradead.org, daniel.lezcano@linaro.org, anna-maria@linutronix.de, kajetan.puchalski@arm.com, lukasz.luba@arm.com, dietmar.eggemann@arm.com, Christian Loehle Subject: [PATCH 4/6] cpuidle: teo: Increase minimum time to stop tick Date: Thu, 6 Jun 2024 10:00:48 +0100 Message-Id: <20240606090050.327614-5-christian.loehle@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240606090050.327614-1-christian.loehle@arm.com> References: <20240606090050.327614-1-christian.loehle@arm.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Since stopping the tick isn't free, add at least some minor constant (1ms) for the threshold to stop the tick. Signed-off-by: Christian Loehle --- drivers/cpuidle/governors/teo.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/cpuidle/governors/teo.c b/drivers/cpuidle/governors/teo.c index 216d34747e3b..ca9422bbd8db 100644 --- a/drivers/cpuidle/governors/teo.c +++ b/drivers/cpuidle/governors/teo.c @@ -622,10 +622,10 @@ static int teo_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, /* * Allow the tick to be stopped unless the selected state is a polling * one or the expected idle duration is shorter than the tick period - * length. + * length plus some constant (1ms) to account for stopping it. */ if ((!(drv->states[idx].flags & CPUIDLE_FLAG_POLLING) && - duration_ns >= TICK_NSEC) || tick_nohz_tick_stopped()) + duration_ns > NSEC_PER_MSEC + TICK_NSEC) || tick_nohz_tick_stopped()) return idx; out_tick_state: From patchwork Thu Jun 6 09:00:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Loehle X-Patchwork-Id: 802226 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id C4500152E00; Thu, 6 Jun 2024 09:01:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717664496; cv=none; b=WartMe6TlDScGQGh81Yxx5TF6xgFfbGZ3gEsqcEWI0Tp2dUgOvI1bCgOdxCxxJIbZJkDms3xEzf6kJZdemI6uNiwg1h0REPKVOtFyBVI8mjRfOtJSkK51yW8WZF/2y9z++1sxxiMhremUWq/Z+e3obMFVXIdkCQz3/nOpGo1z7Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717664496; c=relaxed/simple; bh=+UpKgT0LokTOIBLM1fAipORrgIJz9ctE2y0ChELWHDU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=qMmlAKG4x8TlYj2k4MZp3o4DqW4ZDVjdNz0hsaAZhsFWjt+sHy3sOA5HhzgxZBSt6NdMz+RtWFKjX8HFDG7WOqBi44gQJ0MhEzcamjlyc6QO5NvVMjcJkICiwKBC7+SAWzlTD9M6U4lPq7i4o9utp58fKSH6GkzDurK+U5eC98s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C3647339; Thu, 6 Jun 2024 02:01:58 -0700 (PDT) Received: from e127648.arm.com (unknown [10.57.71.137]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id D7DFE3F762; Thu, 6 Jun 2024 02:01:31 -0700 (PDT) From: Christian Loehle To: linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, rafael@kernel.org Cc: vincent.guittot@linaro.org, qyousef@layalina.io, peterz@infradead.org, daniel.lezcano@linaro.org, anna-maria@linutronix.de, kajetan.puchalski@arm.com, lukasz.luba@arm.com, dietmar.eggemann@arm.com, Christian Loehle Subject: [PATCH 5/6] cpuidle: teo: Remove recent intercepts metric Date: Thu, 6 Jun 2024 10:00:49 +0100 Message-Id: <20240606090050.327614-6-christian.loehle@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240606090050.327614-1-christian.loehle@arm.com> References: <20240606090050.327614-1-christian.loehle@arm.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The logic for recent intercepts didn't work, there is an underflow that can be observed during boot already, which teo usually doesn't recover from, making the entire logic pointless. Furthermore the recent intercepts also were never reset, thus not actually being very 'recent'. If it turns out to be necessary to focus more heavily on resets, the intercepts metric also could be adjusted to decay more quickly. Fixes: 77577558f25d ("cpuidle: teo: Rework most recent idle duration values treatment") Signed-off-by: Christian Loehle --- drivers/cpuidle/governors/teo.c | 74 ++++++--------------------------- 1 file changed, 12 insertions(+), 62 deletions(-) diff --git a/drivers/cpuidle/governors/teo.c b/drivers/cpuidle/governors/teo.c index ca9422bbd8db..3f4801d1e797 100644 --- a/drivers/cpuidle/governors/teo.c +++ b/drivers/cpuidle/governors/teo.c @@ -59,10 +59,6 @@ * shallower than the one whose bin is fallen into by the sleep length (these * situations are referred to as "intercepts" below). * - * In addition to the metrics described above, the governor counts recent - * intercepts (that is, intercepts that have occurred during the last - * %NR_RECENT invocations of it for the given CPU) for each bin. - * * In order to select an idle state for a CPU, the governor takes the following * steps (modulo the possible latency constraint that must be taken into account * too): @@ -81,20 +77,15 @@ * idle long enough to avoid being intercepted if the sleep length had been * equal to the current one). * - * - The sum of the numbers of recent intercepts for all of the idle states - * shallower than the candidate one. - * - * 2. If the second sum is greater than the first one or the third sum is - * greater than %NR_RECENT / 2, the CPU is likely to wake up early, so look - * for an alternative idle state to select. + * 2. If the second sum is greater than the first one the CPU is likely to wake + * up early, so look for an alternative idle state to select. * * - Traverse the idle states shallower than the candidate one in the * descending order. * - * - For each of them compute the sum of the "intercepts" metrics and the sum - * of the numbers of recent intercepts over all of the idle states between - * it and the candidate one (including the former and excluding the - * latter). + * - For each of them compute the sum of the "intercepts" metrics over all + * of the idle states between it and the candidate one (including the + * former and excluding the latter). * * - If each of these sums that needs to be taken into account (because the * check related to it has indicated that the CPU is likely to wake up @@ -159,22 +150,14 @@ #define PULSE 1024 #define DECAY_SHIFT 3 -/* - * Number of the most recent idle duration values to take into consideration for - * the detection of recent early wakeup patterns. - */ -#define NR_RECENT 9 - /** * struct teo_bin - Metrics used by the TEO cpuidle governor. * @intercepts: The "intercepts" metric. * @hits: The "hits" metric. - * @recent: The number of recent "intercepts". */ struct teo_bin { unsigned int intercepts; unsigned int hits; - unsigned int recent; }; /** @@ -183,8 +166,6 @@ struct teo_bin { * @sleep_length_ns: Time till the closest timer event (at the selection time). * @state_bins: Idle state data bins for this CPU. * @total: Grand total of the "intercepts" and "hits" metrics for all bins. - * @next_recent_idx: Index of the next @recent_idx entry to update. - * @recent_idx: Indices of bins corresponding to recent "intercepts". * @tick_hits: Number of "hits" after TICK_NSEC. * @util_threshold: Threshold above which the CPU is considered utilized */ @@ -193,8 +174,6 @@ struct teo_cpu { s64 sleep_length_ns; struct teo_bin state_bins[CPUIDLE_STATE_MAX]; unsigned int total; - int next_recent_idx; - int recent_idx[NR_RECENT]; unsigned int tick_hits; unsigned long util_threshold; }; @@ -284,13 +263,6 @@ static void teo_update(struct cpuidle_driver *drv, struct cpuidle_device *dev) } } - i = cpu_data->next_recent_idx++; - if (cpu_data->next_recent_idx >= NR_RECENT) - cpu_data->next_recent_idx = 0; - - if (cpu_data->recent_idx[i] >= 0) - cpu_data->state_bins[cpu_data->recent_idx[i]].recent--; - /* * If the deepest state's target residency is below the tick length, * make a record of it to help teo_select() decide whether or not @@ -317,14 +289,10 @@ static void teo_update(struct cpuidle_driver *drv, struct cpuidle_device *dev) * Otherwise, update the "intercepts" metric for the bin fallen into by * the measured idle duration. */ - if (idx_timer == idx_duration) { + if (idx_timer == idx_duration) cpu_data->state_bins[idx_timer].hits += PULSE; - cpu_data->recent_idx[i] = -1; - } else { + else cpu_data->state_bins[idx_duration].intercepts += PULSE; - cpu_data->state_bins[idx_duration].recent++; - cpu_data->recent_idx[i] = idx_duration; - } end: cpu_data->total += PULSE; @@ -377,13 +345,10 @@ static int teo_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, unsigned int tick_intercept_sum = 0; unsigned int idx_intercept_sum = 0; unsigned int intercept_sum = 0; - unsigned int idx_recent_sum = 0; - unsigned int recent_sum = 0; unsigned int idx_hit_sum = 0; unsigned int hit_sum = 0; int constraint_idx = 0; int idx0 = 0, idx = -1; - bool alt_intercepts, alt_recent; bool cpu_utilized; s64 duration_ns; int i; @@ -456,7 +421,6 @@ static int teo_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, */ intercept_sum += prev_bin->intercepts; hit_sum += prev_bin->hits; - recent_sum += prev_bin->recent; if (dev->states_usage[i].disable) continue; @@ -472,7 +436,6 @@ static int teo_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, /* Save the sums for the current state. */ idx_intercept_sum = intercept_sum; idx_hit_sum = hit_sum; - idx_recent_sum = recent_sum; } /* Avoid unnecessary overhead. */ @@ -497,37 +460,28 @@ static int teo_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, * If the sum of the intercepts metric for all of the idle states * shallower than the current candidate one (idx) is greater than the * sum of the intercepts and hits metrics for the candidate state and - * all of the deeper states, or the sum of the numbers of recent - * intercepts over all of the states shallower than the candidate one - * is greater than a half of the number of recent events taken into - * account, a shallower idle state is likely to be a better choice. + * all of the deeper states a shallower idle state is likely to be a + * better choice. */ - alt_intercepts = 2 * idx_intercept_sum > cpu_data->total - idx_hit_sum; - alt_recent = idx_recent_sum > NR_RECENT / 2; - if (alt_recent || alt_intercepts) { + if (2 * idx_intercept_sum > cpu_data->total - idx_hit_sum) { int first_suitable_idx = idx; /* * Look for the deepest idle state whose target residency had * not exceeded the idle duration in over a half of the relevant - * cases (both with respect to intercepts overall and with - * respect to the recent intercepts only) in the past. + * cases in the past. * * Take the possible duration limitation present if the tick * has been stopped already into account. */ intercept_sum = 0; - recent_sum = 0; for (i = idx - 1; i >= 0; i--) { struct teo_bin *bin = &cpu_data->state_bins[i]; intercept_sum += bin->intercepts; - recent_sum += bin->recent; - if ((!alt_recent || 2 * recent_sum > idx_recent_sum) && - (!alt_intercepts || - 2 * intercept_sum > idx_intercept_sum)) { + if (2 * intercept_sum > idx_intercept_sum) { /* * Use the current state unless it is too * shallow or disabled, in which case take the @@ -677,15 +631,11 @@ static int teo_enable_device(struct cpuidle_driver *drv, { struct teo_cpu *cpu_data = per_cpu_ptr(&teo_cpus, dev->cpu); unsigned long max_capacity = arch_scale_cpu_capacity(dev->cpu); - int i; memset(cpu_data, 0, sizeof(*cpu_data)); cpu_data->util_threshold = max(UTIL_THRESHOLD_MIN, max_capacity >> UTIL_THRESHOLD_SHIFT); - for (i = 0; i < NR_RECENT; i++) - cpu_data->recent_idx[i] = -1; - return 0; } From patchwork Thu Jun 6 09:00:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christian Loehle X-Patchwork-Id: 802530 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id A306D153804; Thu, 6 Jun 2024 09:01:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717664504; cv=none; b=TSZ2TXgY0j0E2o8bAf4uPVTQRlo6mBG5/jyFJKs1W13NnYO5N7xuNxVJMiV0NuOSBflFN1x2jC4V1+mNnOedp+kbgkANQ1UGzJluHQvgbMtR/Wtz4wrta64jBQWPQVUo5yC84O0MGpfMFIi29g3y7LM+FqHcEbw2IRYUBbWBgog= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717664504; c=relaxed/simple; bh=TpL6sIDxNCxPOYFSOiFcpIYT05cQUxQbaQzGEKLNkG0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nvzo1k5ENbvfyrF6Ri1zSEUXOahsHu77cWf7rUyP428204GN4flTmDxOLDeBITEI3d4cZQBPGCuUhguR+RFVtQ7IPDQc+aK38vZ9zfiyfXwQ6mHHM0rwrf8Vqun1KYnNhl+YkpKq3FCCI7d57QJ8qnQLHwTlTMmCg+6/XYXEfoI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id ADE01339; Thu, 6 Jun 2024 02:02:06 -0700 (PDT) Received: from e127648.arm.com (unknown [10.57.71.137]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 8A29E3F762; Thu, 6 Jun 2024 02:01:39 -0700 (PDT) From: Christian Loehle To: linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, rafael@kernel.org Cc: vincent.guittot@linaro.org, qyousef@layalina.io, peterz@infradead.org, daniel.lezcano@linaro.org, anna-maria@linutronix.de, kajetan.puchalski@arm.com, lukasz.luba@arm.com, dietmar.eggemann@arm.com, Christian Loehle Subject: [PATCH 6/6] cpuidle: teo: Don't count non-existent intercepts Date: Thu, 6 Jun 2024 10:00:50 +0100 Message-Id: <20240606090050.327614-7-christian.loehle@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240606090050.327614-1-christian.loehle@arm.com> References: <20240606090050.327614-1-christian.loehle@arm.com> Precedence: bulk X-Mailing-List: linux-pm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 When bailing out early, teo will not query the sleep length anymore since commit 6da8f9ba5a87 ("cpuidle: teo: Skip tick_nohz_get_sleep_length() call in some cases") with an expected sleep_length_ns value of KTIME_MAX. This lead to state0 accumulating lots of 'intercepts' because the actually measured sleep length was < KTIME_MAX, so count KTIME_MAX as a hit (we have to count them as something otherwise we are stuck). Fundamentally we can only do one of the two: 1. Skip sleep_length_ns query when we think intercept is likely 2. Have accurate data if sleep_length_ns is actually intercepted when we believe it is currently intercepted. This patch chooses that latter as I've found the additional time it takes to query the sleep length to be negligible and the variants of option 1 (count all unknowns as misses or count all unknown as hits) had significant regressions (as misses had lots of too shallow idle state selections and as hits had terrible performance in intercept-heavy workloads). Fixes: 6da8f9ba5a87 ("cpuidle: teo: Skip tick_nohz_get_sleep_length() call in some cases") Signed-off-by: Christian Loehle --- drivers/cpuidle/governors/teo.c | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/drivers/cpuidle/governors/teo.c b/drivers/cpuidle/governors/teo.c index 3f4801d1e797..f41788ff0a94 100644 --- a/drivers/cpuidle/governors/teo.c +++ b/drivers/cpuidle/governors/teo.c @@ -288,8 +288,13 @@ static void teo_update(struct cpuidle_driver *drv, struct cpuidle_device *dev) * length, this is a "hit", so update the "hits" metric for that bin. * Otherwise, update the "intercepts" metric for the bin fallen into by * the measured idle duration. + * If teo_select() bailed out early, we have to count this as a hit as + * we don't know what the true sleep length would've been. Otherwise + * we accumulate lots of intercepts at the shallower state (especially + * state0) even though there weren't any intercepts due to us + * anticipating one. */ - if (idx_timer == idx_duration) + if (idx_timer == idx_duration || cpu_data->sleep_length_ns == KTIME_MAX) cpu_data->state_bins[idx_timer].hits += PULSE; else cpu_data->state_bins[idx_duration].intercepts += PULSE; @@ -349,6 +354,7 @@ static int teo_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, unsigned int hit_sum = 0; int constraint_idx = 0; int idx0 = 0, idx = -1; + int prev_intercept_idx; bool cpu_utilized; s64 duration_ns; int i; @@ -463,6 +469,7 @@ static int teo_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, * all of the deeper states a shallower idle state is likely to be a * better choice. */ + prev_intercept_idx = idx; if (2 * idx_intercept_sum > cpu_data->total - idx_hit_sum) { int first_suitable_idx = idx; @@ -514,6 +521,14 @@ static int teo_select(struct cpuidle_driver *drv, struct cpuidle_device *dev, first_suitable_idx = i; } } + if (prev_intercept_idx != idx && !idx) { + /* + * We have to query the sleep length here otherwise we don't + * know after wakeup if our guess was correct. + */ + duration_ns = tick_nohz_get_sleep_length(&delta_tick); + cpu_data->sleep_length_ns = duration_ns; + } /* * If there is a latency constraint, it may be necessary to select an