From patchwork Tue Aug 26 11:06:44 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 35982 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-pa0-f71.google.com (mail-pa0-f71.google.com [209.85.220.71]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id C3EBA20551 for ; Tue, 26 Aug 2014 11:08:21 +0000 (UTC) Received: by mail-pa0-f71.google.com with SMTP id et14sf122495509pad.6 for ; Tue, 26 Aug 2014 04:08:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:sender:precedence:list-id :x-original-sender:x-original-authentication-results:mailing-list :list-post:list-help:list-archive:list-unsubscribe; bh=ovXisDGo99jzA2rW4elhP+gEoO82Awwp3ls15CDFg1U=; b=Dsk+bj1ENB/STiHc21T7vP5wVpYEmibCjiSld/FxGucaZ1abnsnFRH/9QXq8mnpSpO E6dnfEIA9rI3G+ZI1vbu8m/7/tqzMJ/Q+JrMcOHEXnggi4YjQBa/GFxzGTnQSMBbMwTD QfHkPzUQ+X+K1AWckkQs4+ykWiEEiy70In0hMa8Ac374JTengbQ6jKZJR2eb38ubiUN0 MwxEHBlzG/JAfF5QgTBKRJftnh16+l1dh6Zgt47xMIvrbhnbOaHkFv7CKLPfcmoFJdUY kpqFLhv9gDyQVJXQ90XV6iEKUQxnMIiqriL9W+CGtcItUxtsNuFItIb2qChPHPf9A3R1 A9PA== X-Gm-Message-State: ALoCoQkvkQk5sHBhWR/BsXmnp0PqZ7g9SblI6w9mgQH7EY612RGZBv1cs7TcI7uSeW/174vMEdcU X-Received: by 10.66.224.40 with SMTP id qz8mr18354778pac.20.1409051301077; Tue, 26 Aug 2014 04:08:21 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.107.182 with SMTP id h51ls2553091qgf.80.gmail; Tue, 26 Aug 2014 04:08:20 -0700 (PDT) X-Received: by 10.220.159.204 with SMTP id k12mr23047533vcx.15.1409051300920; Tue, 26 Aug 2014 04:08:20 -0700 (PDT) Received: from mail-vc0-f171.google.com (mail-vc0-f171.google.com [209.85.220.171]) by mx.google.com with ESMTPS id bb7si1207373vdc.58.2014.08.26.04.08.20 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 26 Aug 2014 04:08:20 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.171 as permitted sender) client-ip=209.85.220.171; Received: by mail-vc0-f171.google.com with SMTP id hq11so16631573vcb.16 for ; Tue, 26 Aug 2014 04:08:20 -0700 (PDT) X-Received: by 10.52.156.100 with SMTP id wd4mr9353048vdb.39.1409051300787; Tue, 26 Aug 2014 04:08:20 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.221.45.67 with SMTP id uj3csp189075vcb; Tue, 26 Aug 2014 04:08:20 -0700 (PDT) X-Received: by 10.70.95.34 with SMTP id dh2mr2639263pdb.119.1409051300020; Tue, 26 Aug 2014 04:08:20 -0700 (PDT) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id oy3si3949283pdb.50.2014.08.26.04.08.19 for ; Tue, 26 Aug 2014 04:08:20 -0700 (PDT) Received-SPF: none (google.com: linux-kernel-owner@vger.kernel.org does not designate permitted sender hosts) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757607AbaHZLHu (ORCPT + 26 others); Tue, 26 Aug 2014 07:07:50 -0400 Received: from mail-wg0-f45.google.com ([74.125.82.45]:34796 "EHLO mail-wg0-f45.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755724AbaHZLHr (ORCPT ); Tue, 26 Aug 2014 07:07:47 -0400 Received: by mail-wg0-f45.google.com with SMTP id x12so14589712wgg.16 for ; Tue, 26 Aug 2014 04:07:45 -0700 (PDT) X-Received: by 10.194.202.165 with SMTP id kj5mr2629578wjc.50.1409051265814; Tue, 26 Aug 2014 04:07:45 -0700 (PDT) Received: from lmenx30s.lme.st.com (LPuteaux-656-01-48-212.w82-127.abo.wanadoo.fr. [82.127.83.212]) by mx.google.com with ESMTPSA id q6sm2494891wjy.47.2014.08.26.04.07.43 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 26 Aug 2014 04:07:44 -0700 (PDT) From: Vincent Guittot To: peterz@infradead.org, mingo@kernel.org, linux-kernel@vger.kernel.org, preeti@linux.vnet.ibm.com, linux@arm.linux.org.uk, linux-arm-kernel@lists.infradead.org Cc: riel@redhat.com, Morten.Rasmussen@arm.com, efault@gmx.de, nicolas.pitre@linaro.org, linaro-kernel@lists.linaro.org, daniel.lezcano@linaro.org, dietmar.eggemann@arm.com, Vincent Guittot Subject: [PATCH v5 01/12] sched: fix imbalance flag reset Date: Tue, 26 Aug 2014 13:06:44 +0200 Message-Id: <1409051215-16788-2-git-send-email-vincent.guittot@linaro.org> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1409051215-16788-1-git-send-email-vincent.guittot@linaro.org> References: <1409051215-16788-1-git-send-email-vincent.guittot@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: vincent.guittot@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.171 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , The imbalance flag can stay set whereas there is no imbalance. Let assume that we have 3 tasks that run on a dual cores /dual cluster system. We will have some idle load balance which are triggered during tick. Unfortunately, the tick is also used to queue background work so we can reach the situation where short work has been queued on a CPU which already runs a task. The load balance will detect this imbalance (2 tasks on 1 CPU and an idle CPU) and will try to pull the waiting task on the idle CPU. The waiting task is a worker thread that is pinned on a CPU so an imbalance due to pinned task is detected and the imbalance flag is set. Then, we will not be able to clear the flag because we have at most 1 task on each CPU but the imbalance flag will trig to useless active load balance between the idle CPU and the busy CPU. We need to reset of the imbalance flag as soon as we have reached a balanced state. If all tasks are pinned, we don't consider that as a balanced state and let the imbalance flag set. Reviewed-by: Preeti U Murthy Signed-off-by: Vincent Guittot --- kernel/sched/fair.c | 23 +++++++++++++++++++---- 1 file changed, 19 insertions(+), 4 deletions(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d3427a8..085e853 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6765,10 +6765,8 @@ static int load_balance(int this_cpu, struct rq *this_rq, if (sd_parent) { int *group_imbalance = &sd_parent->groups->sgc->imbalance; - if ((env.flags & LBF_SOME_PINNED) && env.imbalance > 0) { + if ((env.flags & LBF_SOME_PINNED) && env.imbalance > 0) *group_imbalance = 1; - } else if (*group_imbalance) - *group_imbalance = 0; } /* All tasks on this runqueue were pinned by CPU affinity */ @@ -6779,7 +6777,7 @@ static int load_balance(int this_cpu, struct rq *this_rq, env.loop_break = sched_nr_migrate_break; goto redo; } - goto out_balanced; + goto out_all_pinned; } } @@ -6853,6 +6851,23 @@ static int load_balance(int this_cpu, struct rq *this_rq, goto out; out_balanced: + /* + * We reach balance although we may have faced some affinity + * constraints. Clear the imbalance flag if it was set. + */ + if (sd_parent) { + int *group_imbalance = &sd_parent->groups->sgc->imbalance; + + if (*group_imbalance) + *group_imbalance = 0; + } + +out_all_pinned: + /* + * We reach balance because all tasks are pinned at this level so + * we can't migrate them. Let the imbalance flag set so parent level + * can try to migrate them. + */ schedstat_inc(sd, lb_balanced[idle]); sd->nr_balance_failed = 0;