From patchwork Thu Jun 28 15:45:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 140453 Delivered-To: patch@linaro.org Received: by 2002:a2e:9754:0:0:0:0:0 with SMTP id f20-v6csp2342754ljj; Thu, 28 Jun 2018 08:45:27 -0700 (PDT) X-Google-Smtp-Source: ADUXVKKUQb4j+FbXeo/zcW+8e5+KGhnfvvgGnJLYWloXLcAneGpv7lsTyvZU9WDTEtdUACWZ8nCK X-Received: by 2002:a17:902:6b84:: with SMTP id p4-v6mr10994601plk.272.1530200727643; Thu, 28 Jun 2018 08:45:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1530200727; cv=none; d=google.com; s=arc-20160816; b=ZvPXS8FgyO5XFxBNaHvJwdSq9pwrE1amnuzZd2H1xDcgCRAq0lD1q4HPSPD1Z0SNY8 UdolfEOBAhEIuNi1Hjy+PWxo7q1fRU89vyrOEIQ3Pu1CE/kI5gjZ60vL4H3k4x5kFctv jHm0YZM+xSzcn1Bo9KXiiBQOUBo2VvA/g//dN2I+YbOTGzTixtqoc+YjeIFsPJ2hkdNI BnoNwxqc65nJY0DhkiutrogtWJorHL+ckotYShJ543o/+q0aUBAy3m56jcTdvbLXXiQu YGfX1IzyApxFdtnRNt8flxyu9bT8q1soezQrgOhmIFuATzFIkewLVVBCDi0kE4jz2/Zh kosg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=8Ek+YDiDlqnTjlj3EgwbitngcyayYVQm4V9ARSwfL4w=; b=0vthl0omcUY/FPZqBpj2sl+ZOjHbq2pBACamxFzeBpPR3tB38P9fhXIvNvEkuqDhHj d/IfGb3UFvuf3/Z8ew1DhOZDW1/u1uja9N+GWag0QmVzzCX77qr5NJ5MgecfP/4QYBfv DcSKLs3jiZC91MXfAjkNrSTqePuVZ/F1UJ9ot36FHlzFWKszhsziBHTEoIdIPJdGflWn rgV2ShJrQwGND7K4MEEzTnZKDzBri2y+6DDXEvTXKo1+qfxzVpZcpBeY/aAeRmB3zDfH 2UGnMSWxE8aqp1x2Dvp4yTZjL5mtF44r6eBGvhclCrt7kEdfaWkbjMbKdeCw/qocC+zD jANg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kRVPhDOp; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id m75-v6si6827656pfj.192.2018.06.28.08.45.27; Thu, 28 Jun 2018 08:45:27 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kRVPhDOp; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S966947AbeF1PpY (ORCPT + 31 others); Thu, 28 Jun 2018 11:45:24 -0400 Received: from mail-wr0-f195.google.com ([209.85.128.195]:33954 "EHLO mail-wr0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932316AbeF1PpX (ORCPT ); Thu, 28 Jun 2018 11:45:23 -0400 Received: by mail-wr0-f195.google.com with SMTP id a12-v6so6020615wro.1 for ; Thu, 28 Jun 2018 08:45:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=8Ek+YDiDlqnTjlj3EgwbitngcyayYVQm4V9ARSwfL4w=; b=kRVPhDOpSRsVYQaMtgKLcU/gSOU6aIJdlqOJstZIGbRcBDRANEReK+1UCKZy6BmAmm McCqrSlUrrkPAUDBzxlDd+N0YKEFR1v2h0HrvTPLRK4tT26JRHc3Oreul6AMmQI5gZIk BS/NnYr0l4iooduEMzTGFBkYDGT5lZEzSQxhk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=8Ek+YDiDlqnTjlj3EgwbitngcyayYVQm4V9ARSwfL4w=; b=eVbAsjt84JfalOZOzj0SMRh7ZKyUZYQ9fT0D02r5/u/28i1AAJnGJWdbN+vC2nJ0MU L057qJOaJgazBXlkoxHSQAR70EBq17nOCjSLZETZCJOSAvLVVUZYB8XzeqbACC87RdIG DuCcSfXhEZaTh9ryTdI627hkFVeBaPBChJIVqG1XEde+5uxXTYnobMonq49QwiOJcMVW NGbJ4BGtTTSPl00qS95l+gTcZqkDjpumZ84GlT/3MXiKLzawd/TmsdbaojqSYWTP4hl6 m4An9BGCoyPoM40yTMSGbsrke7hy0pptXSt1gn8POeL3CepslQEevcRA8wizv6Ylg91+ agdQ== X-Gm-Message-State: APt69E3OZsu3oHij9u7HLfDIYmpmJbyfEgnSwbLag7mWoKrXaBIvfSiv 0qQYPQU3jATFAQHf1G+y9/+XTA== X-Received: by 2002:a5d:4c4c:: with SMTP id n12-v6mr9004354wrt.71.1530200721550; Thu, 28 Jun 2018 08:45:21 -0700 (PDT) Received: from localhost.localdomain ([2a01:e0a:f:6020:21c3:ec41:bec9:c38]) by smtp.gmail.com with ESMTPSA id i4-v6sm6202115wrq.28.2018.06.28.08.45.20 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 28 Jun 2018 08:45:20 -0700 (PDT) From: Vincent Guittot To: peterz@infradead.org, mingo@kernel.org, linux-kernel@vger.kernel.org Cc: rjw@rjwysocki.net, juri.lelli@redhat.com, dietmar.eggemann@arm.com, Morten.Rasmussen@arm.com, viresh.kumar@linaro.org, valentin.schneider@arm.com, patrick.bellasi@arm.com, joel@joelfernandes.org, daniel.lezcano@linaro.org, quentin.perret@arm.com, luca.abeni@santannapisa.it, claudio@evidence.eu.com, Vincent Guittot Subject: [PATCH v7 00/11] track CPU utilization Date: Thu, 28 Jun 2018 17:45:03 +0200 Message-Id: <1530200714-4504-1-git-send-email-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.7.4 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patchset initially tracked only the utilization of RT rq. During OSPM summit, it has been discussed the opportunity to extend it in order to get an estimate of the utilization of the CPU. - Patches 1 move pelt code in a dedicated file and remove some blank lines - Patches 2-3 add utilization tracking for rt_rq. When both cfs and rt tasks compete to run on a CPU, we can see some frequency drops with schedutil governor. In such case, the cfs_rq's utilization doesn't reflect anymore the utilization of cfs tasks but only the remaining part that is not used by rt tasks. We should monitor the stolen utilization and take it into account when selecting OPP. This patchset doesn't change the OPP selection policy for RT tasks but only for CFS tasks A rt-app use case which creates an always running cfs thread and a rt threads that wakes up periodically with both threads pinned on same CPU, show lot of frequency switches of the CPU whereas the CPU never goes idles during the test. I can share the json file that I used for the test if someone is interested in. For a 15 seconds long test on a hikey 6220 (octo core cortex A53 platfrom), the cpufreq statistics outputs (stats are reset just before the test) : $ cat /sys/devices/system/cpu/cpufreq/policy0/stats/total_trans without patchset : 1230 with patchset : 14 If we replace the cfs thread of rt-app by a sysbench cpu test, we can see performance improvements: - Without patchset : Test execution summary: total time: 15.0009s total number of events: 4903 total time taken by event execution: 14.9972 per-request statistics: min: 1.23ms avg: 3.06ms max: 13.16ms approx. 95 percentile: 12.73ms Threads fairness: events (avg/stddev): 4903.0000/0.00 execution time (avg/stddev): 14.9972/0.00 - With patchset: Test execution summary: total time: 15.0014s total number of events: 7694 total time taken by event execution: 14.9979 per-request statistics: min: 1.23ms avg: 1.95ms max: 10.49ms approx. 95 percentile: 10.39ms Threads fairness: events (avg/stddev): 7694.0000/0.00 execution time (avg/stddev): 14.9979/0.00 The performance improvement is 56% for this use case. - Patches 4-5 add utilization tracking for dl_rq in order to solve similar problem as with rt_rq. Nevertheless, we keep using dl bandwidth as default level of requirement for dl tasks. The dl utilization is used to check that the CPU is not overloaded which is not always reflected when using dl bandwidth - Patches 6-7 add utilization tracking for interrupt and use it select OPP A test with iperf on hikey 6220 gives: w/o patchset w/ patchset Tx 276 Mbits/sec 304 Mbits/sec +10% Rx 299 Mbits/sec 328 Mbits/sec +09% 8 iterations of iperf -c server_address -r -t 5 stdev is lower than 1% Only WFI idle state is enable (shallowest arm idle state) - Patch 8 merges sugov_aggregate_util and sugov_get_util as proposed by Peter - Patches 9 uses rt, dl and interrupt utilization in the scale_rt_capacity() and remove the use of sched_rt_avg_update. - Patches 10 removes the unused sched_avg_update code - Patch 11 removes the unused sched_time_avg_ms Change since v6: - add more comments load tracking metrics - merge sugov_aggregate_util and sugov_get_util Change since v4: - add support of periodic update of blocked utilization - rebase on lastest tip/sched/core Change since v3: - add support of periodic update of blocked utilization - rebase on lastest tip/sched/core Change since v2: - move pelt code into a dedicated pelt.c file - rebase on load tracking changes Change since v1: - Only a rebase. I have addressed the comments on previous version in patch 1/2 Vincent Guittot (11): sched/pelt: Move pelt related code in a dedicated file sched/rt: add rt_rq utilization tracking cpufreq/schedutil: use rt utilization tracking sched/dl: add dl_rq utilization tracking cpufreq/schedutil: use dl utilization tracking sched/irq: add irq utilization tracking cpufreq/schedutil: take into account interrupt sched: schedutil: remove sugov_aggregate_util() sched: use pelt for scale_rt_capacity() sched: remove rt_avg code proc/sched: remove unused sched_time_avg_ms include/linux/sched/sysctl.h | 1 - kernel/sched/Makefile | 2 +- kernel/sched/core.c | 38 +--- kernel/sched/cpufreq_schedutil.c | 65 ++++--- kernel/sched/deadline.c | 8 +- kernel/sched/fair.c | 403 +++++---------------------------------- kernel/sched/pelt.c | 399 ++++++++++++++++++++++++++++++++++++++ kernel/sched/pelt.h | 72 +++++++ kernel/sched/rt.c | 15 +- kernel/sched/sched.h | 68 +++++-- kernel/sysctl.c | 8 - 11 files changed, 632 insertions(+), 447 deletions(-) create mode 100644 kernel/sched/pelt.c create mode 100644 kernel/sched/pelt.h -- 2.7.4 Signed-off-by: Peter Zijlstra (Intel) Acked-by: Viresh Kumar