From patchwork Mon Sep 28 18:35:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Narayan Lal X-Patchwork-Id: 289464 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 483A5C2D0A8 for ; Mon, 28 Sep 2020 18:36:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E6D79207D8 for ; Mon, 28 Sep 2020 18:36:26 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="I0UWEWPB" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726799AbgI1SgV (ORCPT ); Mon, 28 Sep 2020 14:36:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:37741 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726500AbgI1SgU (ORCPT ); Mon, 28 Sep 2020 14:36:20 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601318178; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:in-reply-to:in-reply-to:references:references; bh=iL0bUDw4mzTbRn0mPQir1H2iMDlMMQuuu3RzcEtjrtI=; b=I0UWEWPBKgynHrcG7x2mu74D7cUGGhIH07sUiUIgCvO+RxHn0oECHJcdAz/qykeoZSLXYE dQEbHqQv94xAD9AEYUdOBaEWQnbr9vKu4ugqD68pk4Gcd9653Mo9t/2cXxvIXWGAJJUIMF RhYyiK4uq0U8qoO53vsl3GEvXM6FsAU= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-205-QOoB4A-NNji0hBs9kJXzaw-1; Mon, 28 Sep 2020 14:36:14 -0400 X-MC-Unique: QOoB4A-NNji0hBs9kJXzaw-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 28462ADC25; Mon, 28 Sep 2020 18:36:08 +0000 (UTC) Received: from virtlab719.virt.lab.eng.bos.redhat.com (virtlab719.virt.lab.eng.bos.redhat.com [10.19.153.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4E07860C13; Mon, 28 Sep 2020 18:36:06 +0000 (UTC) From: Nitesh Narayan Lal To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, linux-pci@vger.kernel.org, intel-wired-lan@lists.osuosl.org, frederic@kernel.org, mtosatti@redhat.com, sassmann@redhat.com, jesse.brandeburg@intel.com, lihong.yang@intel.com, helgaas@kernel.org, nitesh@redhat.com, jeffrey.t.kirsher@intel.com, jacob.e.keller@intel.com, jlelli@redhat.com, hch@infradead.org, bhelgaas@google.com, mike.marciniszyn@intel.com, dennis.dalessandro@intel.com, thomas.lendacky@amd.com, jiri@nvidia.com, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, lgoncalv@redhat.com Subject: [PATCH v4 1/4] sched/isolation: API to get number of housekeeping CPUs Date: Mon, 28 Sep 2020 14:35:26 -0400 Message-Id: <20200928183529.471328-2-nitesh@redhat.com> In-Reply-To: <20200928183529.471328-1-nitesh@redhat.com> References: <20200928183529.471328-1-nitesh@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Introduce a new API housekeeping_num_online_cpus(), that can be used to retrieve the number of online housekeeping CPUs based on the housekeeping flag passed by the caller. Some of the consumers for this API are the device drivers that were previously relying only on num_online_cpus() to determine the number of MSIX vectors to create. In real-time environments to minimize interruptions to isolated CPUs, all device-specific IRQ vectors are often moved to the housekeeping CPUs, having excess vectors could cause housekeeping CPU to run out of IRQ vectors. Signed-off-by: Nitesh Narayan Lal --- include/linux/sched/isolation.h | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/include/linux/sched/isolation.h b/include/linux/sched/isolation.h index cc9f393e2a70..e021b1846c1d 100644 --- a/include/linux/sched/isolation.h +++ b/include/linux/sched/isolation.h @@ -57,4 +57,13 @@ static inline bool housekeeping_cpu(int cpu, enum hk_flags flags) return true; } +static inline unsigned int housekeeping_num_online_cpus(enum hk_flags flags) +{ +#ifdef CONFIG_CPU_ISOLATION + if (static_branch_unlikely(&housekeeping_overridden)) + return cpumask_weight(housekeeping_cpumask(flags)); +#endif + return num_online_cpus(); +} + #endif /* _LINUX_SCHED_ISOLATION_H */ From patchwork Mon Sep 28 18:35:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Narayan Lal X-Patchwork-Id: 289462 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 95822C2D0A8 for ; Mon, 28 Sep 2020 18:36:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 47BA0207C4 for ; Mon, 28 Sep 2020 18:36:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="dUns7Tc2" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726775AbgI1SgT (ORCPT ); Mon, 28 Sep 2020 14:36:19 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:29278 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726559AbgI1SgT (ORCPT ); Mon, 28 Sep 2020 14:36:19 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601318177; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:in-reply-to:in-reply-to:references:references; bh=yrgq/KDevcEQJ6jt8mLwCRUIhO0cbeAMHgxfCLcgTj0=; b=dUns7Tc2KtrqusfwwowdxBeYJBRFO8wlF9g3EbA6pxEJd5WVsMc5bhMexKM/CATvbu9mCC 8kXwTMYad5ZgVqgBP6JXokA2wvKri3ssPNPSLy5KUlWuFtIsKPCVIs/XFea5oKruipC363 8MMm1bSjOVKf7+eHjYICSkpnnlSAR3A= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-147-Le-nDnaFMm6JTDFm49LZ4Q-1; Mon, 28 Sep 2020 14:36:15 -0400 X-MC-Unique: Le-nDnaFMm6JTDFm49LZ4Q-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0CCF810BBED3; Mon, 28 Sep 2020 18:36:10 +0000 (UTC) Received: from virtlab719.virt.lab.eng.bos.redhat.com (virtlab719.virt.lab.eng.bos.redhat.com [10.19.153.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 43A4B60C11; Mon, 28 Sep 2020 18:36:08 +0000 (UTC) From: Nitesh Narayan Lal To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, linux-pci@vger.kernel.org, intel-wired-lan@lists.osuosl.org, frederic@kernel.org, mtosatti@redhat.com, sassmann@redhat.com, jesse.brandeburg@intel.com, lihong.yang@intel.com, helgaas@kernel.org, nitesh@redhat.com, jeffrey.t.kirsher@intel.com, jacob.e.keller@intel.com, jlelli@redhat.com, hch@infradead.org, bhelgaas@google.com, mike.marciniszyn@intel.com, dennis.dalessandro@intel.com, thomas.lendacky@amd.com, jiri@nvidia.com, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, lgoncalv@redhat.com Subject: [PATCH v4 2/4] sched/isolation: Extend nohz_full to isolate managed IRQs Date: Mon, 28 Sep 2020 14:35:27 -0400 Message-Id: <20200928183529.471328-3-nitesh@redhat.com> In-Reply-To: <20200928183529.471328-1-nitesh@redhat.com> References: <20200928183529.471328-1-nitesh@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Extend nohz_full feature set to include isolation from managed IRQS. This is required specifically for setups that only uses nohz_full and still requires isolation for maintaining lower latency for the listed CPUs. Suggested-by: Frederic Weisbecker Signed-off-by: Nitesh Narayan Lal --- kernel/sched/isolation.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index 5a6ea03f9882..9df9598a9e39 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -141,7 +141,7 @@ static int __init housekeeping_nohz_full_setup(char *str) unsigned int flags; flags = HK_FLAG_TICK | HK_FLAG_WQ | HK_FLAG_TIMER | HK_FLAG_RCU | - HK_FLAG_MISC | HK_FLAG_KTHREAD; + HK_FLAG_MISC | HK_FLAG_KTHREAD | HK_FLAG_MANAGED_IRQ; return housekeeping_setup(str, flags); } From patchwork Mon Sep 28 18:35:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Narayan Lal X-Patchwork-Id: 260006 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23E67C4727E for ; Mon, 28 Sep 2020 18:36:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D5BA2207C4 for ; Mon, 28 Sep 2020 18:36:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ROBDmRC8" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726859AbgI1Sgi (ORCPT ); Mon, 28 Sep 2020 14:36:38 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:28173 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726801AbgI1SgU (ORCPT ); Mon, 28 Sep 2020 14:36:20 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601318179; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:in-reply-to:in-reply-to:references:references; bh=qD7K+UDAAPv7p1xhR7GsE6Qr4qBNVKTkChHI2rSdLN0=; b=ROBDmRC8mVhOXX8n01XX6+B/1BePrIMcs4cF4NgyrQl24GcQ+nyew5Z7OnMxuRR1SMshBm ZE8TZ+JvN88akOE190Mdk8Uhj3Em7amtJeFQN3HVRXMJAfD40svszvMtjJMu9Y/slQs++W Fp/LOV9NIB6eK5guY1kompMeC2b2h7I= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-378-UZjJXNY8NjeO5agRzp7SQQ-1; Mon, 28 Sep 2020 14:36:14 -0400 X-MC-Unique: UZjJXNY8NjeO5agRzp7SQQ-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E1AE381CBDD; Mon, 28 Sep 2020 18:36:11 +0000 (UTC) Received: from virtlab719.virt.lab.eng.bos.redhat.com (virtlab719.virt.lab.eng.bos.redhat.com [10.19.153.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 261AE60C11; Mon, 28 Sep 2020 18:36:10 +0000 (UTC) From: Nitesh Narayan Lal To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, linux-pci@vger.kernel.org, intel-wired-lan@lists.osuosl.org, frederic@kernel.org, mtosatti@redhat.com, sassmann@redhat.com, jesse.brandeburg@intel.com, lihong.yang@intel.com, helgaas@kernel.org, nitesh@redhat.com, jeffrey.t.kirsher@intel.com, jacob.e.keller@intel.com, jlelli@redhat.com, hch@infradead.org, bhelgaas@google.com, mike.marciniszyn@intel.com, dennis.dalessandro@intel.com, thomas.lendacky@amd.com, jiri@nvidia.com, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, lgoncalv@redhat.com Subject: [PATCH v4 3/4] i40e: Limit msix vectors to housekeeping CPUs Date: Mon, 28 Sep 2020 14:35:28 -0400 Message-Id: <20200928183529.471328-4-nitesh@redhat.com> In-Reply-To: <20200928183529.471328-1-nitesh@redhat.com> References: <20200928183529.471328-1-nitesh@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org If we have isolated CPUs designated to perform real-time tasks, to keep the latency overhead to a minimum for real-time CPUs IRQ vectors are moved to housekeeping CPUs from the userspace. Creating MSIX vectors only based on the online CPUs could lead to exhaustion of housekeeping CPU IRQ vectors in such environments. This patch prevents i40e to create vectors only based on online CPUs by retrieving the online housekeeping CPUs that are designated to perform managed IRQ jobs. Signed-off-by: Nitesh Narayan Lal Reviewed-by: Marcelo Tosatti Acked-by: Jesse Brandeburg --- drivers/net/ethernet/intel/i40e/i40e_main.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/intel/i40e/i40e_main.c b/drivers/net/ethernet/intel/i40e/i40e_main.c index 2e433fdbf2c3..370b581cd48c 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_main.c +++ b/drivers/net/ethernet/intel/i40e/i40e_main.c @@ -5,6 +5,7 @@ #include #include #include +#include #include /* Local includes */ @@ -11002,7 +11003,7 @@ static int i40e_init_msix(struct i40e_pf *pf) * will use any remaining vectors to reach as close as we can to the * number of online CPUs. */ - cpus = num_online_cpus(); + cpus = housekeeping_num_online_cpus(HK_FLAG_MANAGED_IRQ); pf->num_lan_msix = min_t(int, cpus, vectors_left / 2); vectors_left -= pf->num_lan_msix; From patchwork Mon Sep 28 18:35:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nitesh Narayan Lal X-Patchwork-Id: 289463 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.6 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7633C2D0A8 for ; Mon, 28 Sep 2020 18:36:34 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6634B207C4 for ; Mon, 28 Sep 2020 18:36:34 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Ygcw0dGR" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726548AbgI1Sgd (ORCPT ); Mon, 28 Sep 2020 14:36:33 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:60771 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726821AbgI1SgW (ORCPT ); Mon, 28 Sep 2020 14:36:22 -0400 Dkim-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1601318181; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:in-reply-to:in-reply-to:references:references; bh=9oz+ynoTvhiufuVyMQ1xRlRqNWLd2L6kBLmgTWRX5ss=; b=Ygcw0dGRLe/e3XgOOe7GEKbOi33+SmaqafwCUcO1dA69FEVi4WDYJv6GaB8TM3UCP34hBz 9pqAEQf4jHS35m8znmXpPvOXUEhrpWrj7L0SSeSd6impO8FlzNh69wycCKCfriBjA4vW8i fXy/oBzoTgmDSJqjBgXXaYDP5cCQQaA= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-498-vfI2srfRPH2G4PSp-PI-ng-1; Mon, 28 Sep 2020 14:36:16 -0400 X-MC-Unique: vfI2srfRPH2G4PSp-PI-ng-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C434585C732; Mon, 28 Sep 2020 18:36:13 +0000 (UTC) Received: from virtlab719.virt.lab.eng.bos.redhat.com (virtlab719.virt.lab.eng.bos.redhat.com [10.19.153.15]) by smtp.corp.redhat.com (Postfix) with ESMTP id 084E860C11; Mon, 28 Sep 2020 18:36:11 +0000 (UTC) From: Nitesh Narayan Lal To: linux-kernel@vger.kernel.org, netdev@vger.kernel.org, linux-pci@vger.kernel.org, intel-wired-lan@lists.osuosl.org, frederic@kernel.org, mtosatti@redhat.com, sassmann@redhat.com, jesse.brandeburg@intel.com, lihong.yang@intel.com, helgaas@kernel.org, nitesh@redhat.com, jeffrey.t.kirsher@intel.com, jacob.e.keller@intel.com, jlelli@redhat.com, hch@infradead.org, bhelgaas@google.com, mike.marciniszyn@intel.com, dennis.dalessandro@intel.com, thomas.lendacky@amd.com, jiri@nvidia.com, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, lgoncalv@redhat.com Subject: [PATCH v4 4/4] PCI: Limit pci_alloc_irq_vectors() to housekeeping CPUs Date: Mon, 28 Sep 2020 14:35:29 -0400 Message-Id: <20200928183529.471328-5-nitesh@redhat.com> In-Reply-To: <20200928183529.471328-1-nitesh@redhat.com> References: <20200928183529.471328-1-nitesh@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org If we have isolated CPUs dedicated for use by real-time tasks, we try to move IRQs to housekeeping CPUs from the userspace to reduce latency overhead on the isolated CPUs. If we allocate too many IRQ vectors, moving them all to housekeeping CPUs may exceed per-CPU vector limits. When we have isolated CPUs, limit the number of vectors allocated by pci_alloc_irq_vectors() to the minimum number required by the driver, or to one per housekeeping CPU if that is larger. Signed-off-by: Nitesh Narayan Lal Acked-by: Bjorn Helgaas Acked-by: Christoph Hellwig --- drivers/pci/msi.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/drivers/pci/msi.c b/drivers/pci/msi.c index 30ae4ffda5c1..8c156867803c 100644 --- a/drivers/pci/msi.c +++ b/drivers/pci/msi.c @@ -23,6 +23,7 @@ #include #include #include +#include #include "pci.h" @@ -1191,8 +1192,25 @@ int pci_alloc_irq_vectors_affinity(struct pci_dev *dev, unsigned int min_vecs, struct irq_affinity *affd) { struct irq_affinity msi_default_affd = {0}; + unsigned int hk_cpus; int nvecs = -ENOSPC; + hk_cpus = housekeeping_num_online_cpus(HK_FLAG_MANAGED_IRQ); + + /* + * If we have isolated CPUs for use by real-time tasks, to keep the + * latency overhead to a minimum, device-specific IRQ vectors are moved + * to the housekeeping CPUs from the userspace by changing their + * affinity mask. Limit the vector usage to keep housekeeping CPUs from + * running out of IRQ vectors. + */ + if (hk_cpus < num_online_cpus()) { + if (hk_cpus < min_vecs) + max_vecs = min_vecs; + else if (hk_cpus < max_vecs) + max_vecs = hk_cpus; + } + if (flags & PCI_IRQ_AFFINITY) { if (!affd) affd = &msi_default_affd;