From patchwork Wed Aug 26 16:42:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shannon Nelson X-Patchwork-Id: 261894 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59B6AC433E1 for ; Wed, 26 Aug 2020 16:42:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 32A7C2071E for ; Wed, 26 Aug 2020 16:42:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=pensando.io header.i=@pensando.io header.b="41TQ7w3P" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728069AbgHZQmq (ORCPT ); Wed, 26 Aug 2020 12:42:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51142 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727997AbgHZQmZ (ORCPT ); Wed, 26 Aug 2020 12:42:25 -0400 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7AB9C061756 for ; Wed, 26 Aug 2020 09:42:24 -0700 (PDT) Received: by mail-pj1-x1044.google.com with SMTP id kx11so1112000pjb.5 for ; Wed, 26 Aug 2020 09:42:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pensando.io; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=VSnTEpLbazNJA89BLom2epQs6svpK8WKWU4oz2vSyfo=; b=41TQ7w3PErs9xFKwdx5aEnO2CjYTJ0LtGUykB/roIgAwQ2yjGisy/OUobca4tUew7k GBBrnR+ZqBHRhvVLnoFcFEt0wiarYq+A8kFrzh5KkI8KMpUUaiAce4518bTuZtPtuoE5 Ix6ez2J986bbBNkLjGKev7q3ui4MjD2KAgTcs6ZjfFyE9H1viKBM5tep0XgwrqEd0bTA LJu6uL9qPGazqR7B+LN7AC69Pnb29MHBskK0eOKxpBt4rcDN5dmct5Ey5j0jkYEXIKFf QanJjd3Q8/1AUwamr2QSC+savsZk+eHEx6TFGHcyToZyslePj42+TgKLs0N7B1JMwypY +yYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=VSnTEpLbazNJA89BLom2epQs6svpK8WKWU4oz2vSyfo=; b=uXEe2WD93D76fjfybBFCNYA2If1W6kfMuOku7IRTR0nx1d6cCOiapX0PpRmi4pImZz yDg128Sm/dBnXn8ADKi73bo4IYjJ1uFGBDYP8nc2qrnQ5zjs0UpNs3C9/0t7g1ed+nXT ehDvQElTNqD+h4+qy10q3L+KS5rl4hg2emPxP1pXSQ4NJK8z0efaiyVJWzFUKr3nuFsC xe5Fjn7gsucYJeUck3sG+Vl5bQjMzmC24mLyn/bNe/KyNXn/3AIWsx4LdgZna5R3hFRu lQR28RrYVcGcNgcQKwdX1x7/24vAcSJfQrOTyEY79SUycrTWrYRd7KNz4ZC0h+1lInCe BlEw== X-Gm-Message-State: AOAM533qIBxvqaFDLbQ7VqWNh4hERxJ5na5iz9iTyCVRbjVZJEGANN7u fz92TFD7/jXr2dsW6LPx1cVaVoT679Yywg== X-Google-Smtp-Source: ABdhPJzvvyBSDTAo0ZtJ/o73T4d8xe1yect9TA+7QXEwge6KFb6bO5lzPrK9Qp+mT+tIXoMgqXbsBg== X-Received: by 2002:a17:90a:e986:: with SMTP id v6mr110312pjy.167.1598460144004; Wed, 26 Aug 2020 09:42:24 -0700 (PDT) Received: from driver-dev1.pensando.io ([12.226.153.42]) by smtp.gmail.com with ESMTPSA id h193sm2986052pgc.42.2020.08.26.09.42.23 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Aug 2020 09:42:23 -0700 (PDT) From: Shannon Nelson To: netdev@vger.kernel.org, davem@davemloft.net Cc: Shannon Nelson Subject: [PATCH net-next 02/12] ionic: fix up a couple of debug strings Date: Wed, 26 Aug 2020 09:42:04 -0700 Message-Id: <20200826164214.31792-3-snelson@pensando.io> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826164214.31792-1-snelson@pensando.io> References: <20200826164214.31792-1-snelson@pensando.io> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Fix the queue name displayed. Signed-off-by: Shannon Nelson --- drivers/net/ethernet/pensando/ionic/ionic_lif.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index 235215c28f29..e95e3fa8840a 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -433,14 +433,14 @@ static int ionic_qcq_alloc(struct ionic_lif *lif, unsigned int type, err = ionic_intr_alloc(lif, &new->intr); if (err) { netdev_warn(lif->netdev, "no intr for %s: %d\n", - name, err); + new->q.name, err); goto err_out; } err = ionic_bus_get_irq(lif->ionic, new->intr.index); if (err < 0) { netdev_warn(lif->netdev, "no vector for %s: %d\n", - name, err); + new->q.name, err); goto err_out_free_intr; } new->intr.vector = err; @@ -449,7 +449,8 @@ static int ionic_qcq_alloc(struct ionic_lif *lif, unsigned int type, err = ionic_request_irq(lif, new); if (err) { - netdev_warn(lif->netdev, "irq request failed %d\n", err); + netdev_warn(lif->netdev, "irq request failed for %s: %d\n", + new->q.name, err); goto err_out_free_intr; } From patchwork Wed Aug 26 16:42:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shannon Nelson X-Patchwork-Id: 261893 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 50E28C433E1 for ; Wed, 26 Aug 2020 16:43:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 031E2206FA for ; Wed, 26 Aug 2020 16:43:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=pensando.io header.i=@pensando.io header.b="otmH89k+" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728103AbgHZQm7 (ORCPT ); Wed, 26 Aug 2020 12:42:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51156 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728005AbgHZQm1 (ORCPT ); Wed, 26 Aug 2020 12:42:27 -0400 Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2DEF9C0613ED for ; Wed, 26 Aug 2020 09:42:27 -0700 (PDT) Received: by mail-pg1-x541.google.com with SMTP id g33so1320888pgb.4 for ; Wed, 26 Aug 2020 09:42:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pensando.io; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=R/ib6Ww1+kDrDThZznQQ5PG5P20/sGy3hQifxbMCpwI=; b=otmH89k+uki+rkxLLYXH0IZWr8i+nK1NpLrt0vnfX6WIhZwY1qa7LOqqtdgFV5EOS4 yXvaC8v2z7AabGqek5Mw6cjHkSZ2P0LjMXHVbAKrJ1DXjQKkoYx3KWmZtOvRe6ZlYwJj 2TwSps2YLgjPeaKFulRrkq5vS13GOwQgy2NGZ/4Agr6/5qxltlVYp6Jg3E196ck/oYYR hf1k7D8Jp6GJy2muforrfAbwJSJ/OQTOC7CzgoUXAgrcMhh4yFREW3GBtgAKC+h2HqD1 Z9ncyJLPDoykpMoJ0V5OcxLVR+OmwsBv6rZeVxBqnj0AlCL5lOwKCxOw+SYBq0CxBDiu dp9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=R/ib6Ww1+kDrDThZznQQ5PG5P20/sGy3hQifxbMCpwI=; b=NXwJgCsFJS2e0VxeGDC1EUz0sKgf5y039Wkd3W3a9m6+ryC943Kru72T7Tr02sk4h+ igNZtrSPTHKtHHym69Eohj6NYNu4cXCNS7ZFAzzLKgUEiAUILkxuGsvj5Y9vFqT3zm1W q0FM4HDrNpJUB0BSBOWOSoYIfNuzbaDS+nrazOYmRNBbMfQrdKry2bo3Xfhh942ddUqx jYgx+jiXcMfPGdhAqzN+gdvEO0T4cJBJNChdIfN+DDCSkvp1rJcvCcSp6pV7hkgjE4xL Gnpy+6/a/h6HD+PqHzREJa3dWhpWolflfj14/XObCsMgF7efpQmvuT/oG0yTDaMeb3XD FHSw== X-Gm-Message-State: AOAM533gRi3q9/Eqs3pch60mDhk4gfyRy5zTqmZ8mfJW/KwjhsuLFJTe BmXJbsVKXJM9bOLf4+Erhi5XjcjEpu3zFg== X-Google-Smtp-Source: ABdhPJwl8HBoTsZ3C+SatpQ6VhTH5SXxMRxdQ54gXZ3P9yl8n4bAsGgHxM49D5vaS1EU5Tlohvi5Qg== X-Received: by 2002:a65:45ca:: with SMTP id m10mr4966190pgr.187.1598460146204; Wed, 26 Aug 2020 09:42:26 -0700 (PDT) Received: from driver-dev1.pensando.io ([12.226.153.42]) by smtp.gmail.com with ESMTPSA id h193sm2986052pgc.42.2020.08.26.09.42.25 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Aug 2020 09:42:25 -0700 (PDT) From: Shannon Nelson To: netdev@vger.kernel.org, davem@davemloft.net Cc: Shannon Nelson Subject: [PATCH net-next 04/12] ionic: remove lif list concept Date: Wed, 26 Aug 2020 09:42:06 -0700 Message-Id: <20200826164214.31792-5-snelson@pensando.io> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826164214.31792-1-snelson@pensando.io> References: <20200826164214.31792-1-snelson@pensando.io> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org As we aren't yet supporting multiple lifs, we can remove complexity by removing the list concept and related code, to be re-engineered later when actually needed. Signed-off-by: Shannon Nelson --- drivers/net/ethernet/pensando/ionic/ionic.h | 4 +- .../ethernet/pensando/ionic/ionic_bus_pci.c | 32 +++-- .../net/ethernet/pensando/ionic/ionic_dev.c | 6 +- .../ethernet/pensando/ionic/ionic_devlink.c | 2 +- .../net/ethernet/pensando/ionic/ionic_lif.c | 131 +++++------------- .../net/ethernet/pensando/ionic/ionic_lif.h | 14 +- 6 files changed, 62 insertions(+), 127 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic.h b/drivers/net/ethernet/pensando/ionic/ionic.h index f5a910c458ba..f699ed19eb4f 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic.h +++ b/drivers/net/ethernet/pensando/ionic/ionic.h @@ -42,13 +42,11 @@ struct ionic { struct ionic_dev_bar bars[IONIC_BARS_MAX]; unsigned int num_bars; struct ionic_identity ident; - struct list_head lifs; - struct ionic_lif *master_lif; + struct ionic_lif *lif; unsigned int nnqs_per_lif; unsigned int neqs_per_lif; unsigned int ntxqs_per_lif; unsigned int nrxqs_per_lif; - DECLARE_BITMAP(lifbits, IONIC_LIFS_MAX); unsigned int nintrs; DECLARE_BITMAP(intrs, IONIC_INTR_CTRL_REGS_MAX); struct work_struct nb_work; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c b/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c index 85c686c16741..d1d6fb6669e5 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c @@ -294,21 +294,21 @@ static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) goto err_out_port_reset; } - err = ionic_lifs_size(ionic); + err = ionic_lif_size(ionic); if (err) { - dev_err(dev, "Cannot size LIFs: %d, aborting\n", err); + dev_err(dev, "Cannot size LIF: %d, aborting\n", err); goto err_out_port_reset; } - err = ionic_lifs_alloc(ionic); + err = ionic_lif_alloc(ionic); if (err) { - dev_err(dev, "Cannot allocate LIFs: %d, aborting\n", err); + dev_err(dev, "Cannot allocate LIF: %d, aborting\n", err); goto err_out_free_irqs; } - err = ionic_lifs_init(ionic); + err = ionic_lif_init(ionic->lif); if (err) { - dev_err(dev, "Cannot init LIFs: %d, aborting\n", err); + dev_err(dev, "Cannot init LIF: %d, aborting\n", err); goto err_out_free_lifs; } @@ -321,9 +321,9 @@ static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) dev_err(dev, "Cannot enable existing VFs: %d\n", err); } - err = ionic_lifs_register(ionic); + err = ionic_lif_register(ionic->lif); if (err) { - dev_err(dev, "Cannot register LIFs: %d, aborting\n", err); + dev_err(dev, "Cannot register LIF: %d, aborting\n", err); goto err_out_deinit_lifs; } @@ -336,12 +336,13 @@ static int ionic_probe(struct pci_dev *pdev, const struct pci_device_id *ent) return 0; err_out_deregister_lifs: - ionic_lifs_unregister(ionic); + ionic_lif_unregister(ionic->lif); err_out_deinit_lifs: ionic_vf_dealloc(ionic); - ionic_lifs_deinit(ionic); + ionic_lif_deinit(ionic->lif); err_out_free_lifs: - ionic_lifs_free(ionic); + ionic_lif_free(ionic->lif); + ionic->lif = NULL; err_out_free_irqs: ionic_bus_free_irq_vectors(ionic); err_out_port_reset: @@ -377,11 +378,12 @@ static void ionic_remove(struct pci_dev *pdev) if (!ionic) return; - if (ionic->master_lif) { + if (ionic->lif) { ionic_devlink_unregister(ionic); - ionic_lifs_unregister(ionic); - ionic_lifs_deinit(ionic); - ionic_lifs_free(ionic); + ionic_lif_unregister(ionic->lif); + ionic_lif_deinit(ionic->lif); + ionic_lif_free(ionic->lif); + ionic->lif = NULL; ionic_bus_free_irq_vectors(ionic); } diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.c b/drivers/net/ethernet/pensando/ionic/ionic_dev.c index d83eff0ae0ac..25cf376f3b40 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_dev.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.c @@ -21,8 +21,8 @@ static void ionic_watchdog_cb(struct timer_list *t) hb = ionic_heartbeat_check(ionic); - if (hb >= 0 && ionic->master_lif) - ionic_link_status_check_request(ionic->master_lif); + if (hb >= 0 && ionic->lif) + ionic_link_status_check_request(ionic->lif); } void ionic_init_devinfo(struct ionic *ionic) @@ -126,7 +126,7 @@ int ionic_heartbeat_check(struct ionic *ionic) /* is this a transition? */ if (fw_status != idev->last_fw_status && idev->last_fw_status != 0xff) { - struct ionic_lif *lif = ionic->master_lif; + struct ionic_lif *lif = ionic->lif; bool trigger = false; if (!fw_status || fw_status == 0xff) { diff --git a/drivers/net/ethernet/pensando/ionic/ionic_devlink.c b/drivers/net/ethernet/pensando/ionic/ionic_devlink.c index c4f4fd469fe3..8d9fb2e19cca 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_devlink.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_devlink.c @@ -85,7 +85,7 @@ int ionic_devlink_register(struct ionic *ionic) dev_err(ionic->dev, "devlink_port_register failed: %d\n", err); else devlink_port_type_eth_set(&ionic->dl_port, - ionic->master_lif->netdev); + ionic->lif->netdev); return err; } diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index d73beddc30cc..de9da16db3a8 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -2045,7 +2045,7 @@ int ionic_reset_queues(struct ionic_lif *lif, ionic_reset_cb cb, void *arg) return err; } -static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index) +int ionic_lif_alloc(struct ionic *ionic) { struct device *dev = ionic->dev; union ionic_lif_identity *lid; @@ -2056,7 +2056,7 @@ static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index lid = kzalloc(sizeof(*lid), GFP_KERNEL); if (!lid) - return ERR_PTR(-ENOMEM); + return -ENOMEM; netdev = alloc_etherdev_mqs(sizeof(*lif), ionic->ntxqs_per_lif, ionic->ntxqs_per_lif); @@ -2070,7 +2070,7 @@ static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index lif = netdev_priv(netdev); lif->netdev = netdev; - ionic->master_lif = lif; + ionic->lif = lif; netdev->netdev_ops = &ionic_netdev_ops; ionic_ethtool_set_ops(netdev); @@ -2089,7 +2089,7 @@ static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index lif->nxqs = ionic->ntxqs_per_lif; lif->ionic = ionic; - lif->index = index; + lif->index = 0; lif->ntxq_descs = IONIC_DEF_TXRX_DESC; lif->nrxq_descs = IONIC_DEF_TXRX_DESC; lif->tx_budget = IONIC_TX_BUDGET_DEFAULT; @@ -2101,7 +2101,7 @@ static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index lif->tx_coalesce_usecs = lif->rx_coalesce_usecs; lif->tx_coalesce_hw = lif->rx_coalesce_hw; - snprintf(lif->name, sizeof(lif->name), "lif%u", index); + snprintf(lif->name, sizeof(lif->name), "lif%u", lif->index); spin_lock_init(&lif->adminq_lock); @@ -2121,7 +2121,8 @@ static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index ionic_debugfs_add_lif(lif); - /* allocate queues */ + /* allocate control queues and txrx queue arrays */ + ionic_lif_queue_identify(lif); err = ionic_qcqs_alloc(lif); if (err) goto err_out_free_lif_info; @@ -2140,9 +2141,7 @@ static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index } netdev_rss_key_fill(lif->rss_hash_key, IONIC_RSS_HASH_KEY_SIZE); - list_add_tail(&lif->list, &ionic->lifs); - - return lif; + return 0; err_out_free_qcqs: ionic_qcqs_free(lif); @@ -2156,27 +2155,7 @@ static struct ionic_lif *ionic_lif_alloc(struct ionic *ionic, unsigned int index err_out_free_lid: kfree(lid); - return ERR_PTR(err); -} - -int ionic_lifs_alloc(struct ionic *ionic) -{ - struct ionic_lif *lif; - - INIT_LIST_HEAD(&ionic->lifs); - - /* only build the first lif, others are for later features */ - set_bit(0, ionic->lifbits); - - lif = ionic_lif_alloc(ionic, 0); - if (IS_ERR_OR_NULL(lif)) { - clear_bit(0, ionic->lifbits); - return -ENOMEM; - } - - ionic_lif_queue_identify(lif); - - return 0; + return err; } static void ionic_lif_reset(struct ionic_lif *lif) @@ -2211,7 +2190,7 @@ static void ionic_lif_handle_fw_down(struct ionic_lif *lif) ionic_txrx_deinit(lif); ionic_txrx_free(lif); } - ionic_lifs_deinit(ionic); + ionic_lif_deinit(lif); ionic_reset(ionic); ionic_qcqs_free(lif); @@ -2234,7 +2213,7 @@ static void ionic_lif_handle_fw_up(struct ionic_lif *lif) if (err) goto err_out; - err = ionic_lifs_init(ionic); + err = ionic_lif_init(lif); if (err) goto err_qcqs_free; @@ -2263,14 +2242,14 @@ static void ionic_lif_handle_fw_up(struct ionic_lif *lif) err_txrx_free: ionic_txrx_free(lif); err_lifs_deinit: - ionic_lifs_deinit(ionic); + ionic_lif_deinit(lif); err_qcqs_free: ionic_qcqs_free(lif); err_out: dev_err(ionic->dev, "FW Up: LIFs restart failed - err %d\n", err); } -static void ionic_lif_free(struct ionic_lif *lif) +void ionic_lif_free(struct ionic_lif *lif) { struct device *dev = lif->ionic->dev; @@ -2299,23 +2278,10 @@ static void ionic_lif_free(struct ionic_lif *lif) /* free netdev & lif */ ionic_debugfs_del_lif(lif); - list_del(&lif->list); free_netdev(lif->netdev); } -void ionic_lifs_free(struct ionic *ionic) -{ - struct list_head *cur, *tmp; - struct ionic_lif *lif; - - list_for_each_safe(cur, tmp, &ionic->lifs) { - lif = list_entry(cur, struct ionic_lif, list); - - ionic_lif_free(lif); - } -} - -static void ionic_lif_deinit(struct ionic_lif *lif) +void ionic_lif_deinit(struct ionic_lif *lif) { if (!test_and_clear_bit(IONIC_LIF_F_INITED, lif->state)) return; @@ -2336,17 +2302,6 @@ static void ionic_lif_deinit(struct ionic_lif *lif) ionic_lif_reset(lif); } -void ionic_lifs_deinit(struct ionic *ionic) -{ - struct list_head *cur, *tmp; - struct ionic_lif *lif; - - list_for_each_safe(cur, tmp, &ionic->lifs) { - lif = list_entry(cur, struct ionic_lif, list); - ionic_lif_deinit(lif); - } -} - static int ionic_lif_adminq_init(struct ionic_lif *lif) { struct device *dev = lif->ionic->dev; @@ -2492,7 +2447,7 @@ static int ionic_station_set(struct ionic_lif *lif) return 0; } -static int ionic_lif_init(struct ionic_lif *lif) +int ionic_lif_init(struct ionic_lif *lif) { struct ionic_dev *idev = &lif->ionic->idev; struct device *dev = lif->ionic->dev; @@ -2582,22 +2537,6 @@ static int ionic_lif_init(struct ionic_lif *lif) return err; } -int ionic_lifs_init(struct ionic *ionic) -{ - struct list_head *cur, *tmp; - struct ionic_lif *lif; - int err; - - list_for_each_safe(cur, tmp, &ionic->lifs) { - lif = list_entry(cur, struct ionic_lif, list); - err = ionic_lif_init(lif); - if (err) - return err; - } - - return 0; -} - static void ionic_lif_notify_work(struct work_struct *ws) { } @@ -2646,45 +2585,41 @@ static int ionic_lif_notify(struct notifier_block *nb, return NOTIFY_DONE; } -int ionic_lifs_register(struct ionic *ionic) +int ionic_lif_register(struct ionic_lif *lif) { int err; - INIT_WORK(&ionic->nb_work, ionic_lif_notify_work); + INIT_WORK(&lif->ionic->nb_work, ionic_lif_notify_work); - ionic->nb.notifier_call = ionic_lif_notify; + lif->ionic->nb.notifier_call = ionic_lif_notify; - err = register_netdevice_notifier(&ionic->nb); + err = register_netdevice_notifier(&lif->ionic->nb); if (err) - ionic->nb.notifier_call = NULL; + lif->ionic->nb.notifier_call = NULL; /* only register LIF0 for now */ - err = register_netdev(ionic->master_lif->netdev); + err = register_netdev(lif->netdev); if (err) { - dev_err(ionic->dev, "Cannot register net device, aborting\n"); + dev_err(lif->ionic->dev, "Cannot register net device, aborting\n"); return err; } - ionic->master_lif->registered = true; - ionic_lif_set_netdev_info(ionic->master_lif); + lif->registered = true; + ionic_lif_set_netdev_info(lif); return 0; } -void ionic_lifs_unregister(struct ionic *ionic) +void ionic_lif_unregister(struct ionic_lif *lif) { - if (ionic->nb.notifier_call) { - unregister_netdevice_notifier(&ionic->nb); - cancel_work_sync(&ionic->nb_work); - ionic->nb.notifier_call = NULL; + if (lif->ionic->nb.notifier_call) { + unregister_netdevice_notifier(&lif->ionic->nb); + cancel_work_sync(&lif->ionic->nb_work); + lif->ionic->nb.notifier_call = NULL; } - /* There is only one lif ever registered in the - * current model, so don't bother searching the - * ionic->lif for candidates to unregister - */ - if (ionic->master_lif && - ionic->master_lif->netdev->reg_state == NETREG_REGISTERED) - unregister_netdev(ionic->master_lif->netdev); + if (lif->netdev->reg_state == NETREG_REGISTERED) + unregister_netdev(lif->netdev); + lif->registered = false; } static void ionic_lif_queue_identify(struct ionic_lif *lif) @@ -2803,7 +2738,7 @@ int ionic_lif_identify(struct ionic *ionic, u8 lif_type, return 0; } -int ionic_lifs_size(struct ionic *ionic) +int ionic_lif_size(struct ionic *ionic) { struct ionic_identity *ident = &ionic->ident; unsigned int nintrs, dev_nintrs; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.h b/drivers/net/ethernet/pensando/ionic/ionic_lif.h index 1ee3b14c8d50..95d85502c18d 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.h @@ -242,15 +242,15 @@ void ionic_get_stats64(struct net_device *netdev, struct rtnl_link_stats64 *ns); void ionic_lif_deferred_enqueue(struct ionic_deferred *def, struct ionic_deferred_work *work); -int ionic_lifs_alloc(struct ionic *ionic); -void ionic_lifs_free(struct ionic *ionic); -void ionic_lifs_deinit(struct ionic *ionic); -int ionic_lifs_init(struct ionic *ionic); -int ionic_lifs_register(struct ionic *ionic); -void ionic_lifs_unregister(struct ionic *ionic); +int ionic_lif_alloc(struct ionic *ionic); +int ionic_lif_init(struct ionic_lif *lif); +void ionic_lif_free(struct ionic_lif *lif); +void ionic_lif_deinit(struct ionic_lif *lif); +int ionic_lif_register(struct ionic_lif *lif); +void ionic_lif_unregister(struct ionic_lif *lif); int ionic_lif_identify(struct ionic *ionic, u8 lif_type, union ionic_lif_identity *lif_ident); -int ionic_lifs_size(struct ionic *ionic); +int ionic_lif_size(struct ionic *ionic); int ionic_lif_rss_config(struct ionic_lif *lif, u16 types, const u8 *key, const u32 *indir); From patchwork Wed Aug 26 16:42:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shannon Nelson X-Patchwork-Id: 261892 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57F6FC433E1 for ; Wed, 26 Aug 2020 16:43:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1276D206FA for ; Wed, 26 Aug 2020 16:43:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=pensando.io header.i=@pensando.io header.b="leIjg/hH" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728127AbgHZQnk (ORCPT ); Wed, 26 Aug 2020 12:43:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51168 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727017AbgHZQmb (ORCPT ); Wed, 26 Aug 2020 12:42:31 -0400 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EB8CBC061786 for ; Wed, 26 Aug 2020 09:42:30 -0700 (PDT) Received: by mail-pl1-x642.google.com with SMTP id bh1so1150399plb.12 for ; Wed, 26 Aug 2020 09:42:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pensando.io; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UBkhLW7LLWjhplLSLvn0LAzLdJdw+e5t6cqPLAbVNG0=; b=leIjg/hHbM854/I1ngFPMsU1KXe4RfwganY/n91dEkfk+SoPkXxhuGEdstH+NiorQq 5GOb6dnBq6eRUHRP+jWkhIl3O7FC3rCDUVUuWM+14sLHlU+eFDTZyh2DcAfagTao73Oh 1iLbswHB6nfvomcYkEkOcRCATQrBQhX/OEL9ZM375d4GnkJvMEW1cQdAALI2r3AgsXk7 IZ8dJM1hd4MXrhEAU23f4hsswQF3PSnUGQb3w20JACuoH32E509Fw9hL83i3JcNGG9Ah vKrG4LISuuzu7phl/JjB8ASkgDNtZ7jwOieO0JfPx83OpspcBibFGJqc7CuY/1QQtIuD hWFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UBkhLW7LLWjhplLSLvn0LAzLdJdw+e5t6cqPLAbVNG0=; b=BPoArdh4KW0L5fCNHVgcklcdaUcG+4HDV6OnOER3FqH8Z9Z/kllGlRbtXcBBALP8FX DlRa8roNfFH+KLZh91xZ84h4iORhXGbNOvMKSY1/xeQM3wJRWEozjdAQCtiOF1WHRp6u WtY7G/Phlzn2S0GT72f9uNYNwWDjDOMl4hwojubYYM2u7QMk0Q30d4ZLkdNTYRVMWemU meqUSXMiaRlOz2DuCwD1DEO57xjkp5ZxR9w0jk5JAiIfApwBl8Jy0xxkUniKG6fsSi+A Oj6btwfr54g0C/8W4yxN1b7K/TTmx44gYxjmMxfen2fwhgUJiC36ep1eAOAkI/XuOqvn 7oJA== X-Gm-Message-State: AOAM5304ZHAiVEr0qsKzRuY71eRQsCwcZCji1TJcnV1D+TbdkKzglZ6U vcfx+QHu/jH9309lABuZ1uld9Rd4dlJLYQ== X-Google-Smtp-Source: ABdhPJw2L+40WwQbHCw5DPI4LX3J8F2dUD9849wbW9tZqkM+kdbKYAiH5XqGwK7eRvBi9qgjCzOATw== X-Received: by 2002:a17:90b:1093:: with SMTP id gj19mr6949472pjb.149.1598460149857; Wed, 26 Aug 2020 09:42:29 -0700 (PDT) Received: from driver-dev1.pensando.io ([12.226.153.42]) by smtp.gmail.com with ESMTPSA id h193sm2986052pgc.42.2020.08.26.09.42.28 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Aug 2020 09:42:29 -0700 (PDT) From: Shannon Nelson To: netdev@vger.kernel.org, davem@davemloft.net Cc: Shannon Nelson Subject: [PATCH net-next 07/12] ionic: reduce contiguous memory allocation requirement Date: Wed, 26 Aug 2020 09:42:09 -0700 Message-Id: <20200826164214.31792-8-snelson@pensando.io> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826164214.31792-1-snelson@pensando.io> References: <20200826164214.31792-1-snelson@pensando.io> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Split out the queue descriptor blocks into separate dma allocations to make for smaller blocks. Authored-by: Neel Patel Signed-off-by: Shannon Nelson --- .../ethernet/pensando/ionic/ionic_debugfs.c | 23 ++-- .../net/ethernet/pensando/ionic/ionic_lif.c | 100 +++++++++++------- .../net/ethernet/pensando/ionic/ionic_lif.h | 12 ++- 3 files changed, 81 insertions(+), 54 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_debugfs.c b/drivers/net/ethernet/pensando/ionic/ionic_debugfs.c index 11621ccc1faf..e12dbe4ea73d 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_debugfs.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_debugfs.c @@ -112,7 +112,8 @@ static const struct debugfs_reg32 intr_ctrl_regs[] = { void ionic_debugfs_add_qcq(struct ionic_lif *lif, struct ionic_qcq *qcq) { - struct dentry *q_dentry, *cq_dentry, *intr_dentry, *stats_dentry; + struct dentry *qcq_dentry, *q_dentry, *cq_dentry; + struct dentry *intr_dentry, *stats_dentry; struct ionic_dev *idev = &lif->ionic->idev; struct debugfs_regset32 *intr_ctrl_regset; struct ionic_intr_info *intr = &qcq->intr; @@ -121,21 +122,21 @@ void ionic_debugfs_add_qcq(struct ionic_lif *lif, struct ionic_qcq *qcq) struct ionic_queue *q = &qcq->q; struct ionic_cq *cq = &qcq->cq; - qcq->dentry = debugfs_create_dir(q->name, lif->dentry); + qcq_dentry = debugfs_create_dir(q->name, lif->dentry); + if (IS_ERR_OR_NULL(qcq_dentry)) + return; + qcq->dentry = qcq_dentry; - debugfs_create_x32("total_size", 0400, qcq->dentry, &qcq->total_size); - debugfs_create_x64("base_pa", 0400, qcq->dentry, &qcq->base_pa); + debugfs_create_x64("q_base_pa", 0400, qcq_dentry, &qcq->q_base_pa); + debugfs_create_x32("q_size", 0400, qcq_dentry, &qcq->q_size); + debugfs_create_x64("cq_base_pa", 0400, qcq_dentry, &qcq->cq_base_pa); + debugfs_create_x32("cq_size", 0400, qcq_dentry, &qcq->cq_size); + debugfs_create_x64("sg_base_pa", 0400, qcq_dentry, &qcq->sg_base_pa); + debugfs_create_x32("sg_size", 0400, qcq_dentry, &qcq->sg_size); q_dentry = debugfs_create_dir("q", qcq->dentry); debugfs_create_u32("index", 0400, q_dentry, &q->index); - debugfs_create_x64("base_pa", 0400, q_dentry, &q->base_pa); - if (qcq->flags & IONIC_QCQ_F_SG) { - debugfs_create_x64("sg_base_pa", 0400, q_dentry, - &q->sg_base_pa); - debugfs_create_u32("sg_desc_size", 0400, q_dentry, - &q->sg_desc_size); - } debugfs_create_u32("num_descs", 0400, q_dentry, &q->num_descs); debugfs_create_u32("desc_size", 0400, q_dentry, &q->desc_size); debugfs_create_u32("pid", 0400, q_dentry, &q->pid); diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index 6a50bb6f090c..016d55ad1f6a 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -306,9 +306,23 @@ static void ionic_qcq_free(struct ionic_lif *lif, struct ionic_qcq *qcq) ionic_debugfs_del_qcq(qcq); - dma_free_coherent(dev, qcq->total_size, qcq->base, qcq->base_pa); - qcq->base = NULL; - qcq->base_pa = 0; + if (qcq->q_base) { + dma_free_coherent(dev, qcq->q_size, qcq->q_base, qcq->q_base_pa); + qcq->q_base = NULL; + qcq->q_base_pa = 0; + } + + if (qcq->cq_base) { + dma_free_coherent(dev, qcq->cq_size, qcq->cq_base, qcq->cq_base_pa); + qcq->cq_base = NULL; + qcq->cq_base_pa = 0; + } + + if (qcq->sg_base) { + dma_free_coherent(dev, qcq->sg_size, qcq->sg_base, qcq->sg_base_pa); + qcq->sg_base = NULL; + qcq->sg_base_pa = 0; + } if (qcq->flags & IONIC_QCQ_F_INTR) { irq_set_affinity_hint(qcq->intr.vector, NULL); @@ -374,7 +388,6 @@ static int ionic_qcq_alloc(struct ionic_lif *lif, unsigned int type, unsigned int pid, struct ionic_qcq **qcq) { struct ionic_dev *idev = &lif->ionic->idev; - u32 q_size, cq_size, sg_size, total_size; struct device *dev = lif->ionic->dev; void *q_base, *cq_base, *sg_base; dma_addr_t cq_base_pa = 0; @@ -385,21 +398,6 @@ static int ionic_qcq_alloc(struct ionic_lif *lif, unsigned int type, *qcq = NULL; - q_size = num_descs * desc_size; - cq_size = num_descs * cq_desc_size; - sg_size = num_descs * sg_desc_size; - - total_size = ALIGN(q_size, PAGE_SIZE) + ALIGN(cq_size, PAGE_SIZE); - /* Note: aligning q_size/cq_size is not enough due to cq_base - * address aligning as q_base could be not aligned to the page. - * Adding PAGE_SIZE. - */ - total_size += PAGE_SIZE; - if (flags & IONIC_QCQ_F_SG) { - total_size += ALIGN(sg_size, PAGE_SIZE); - total_size += PAGE_SIZE; - } - new = devm_kzalloc(dev, sizeof(*new), GFP_KERNEL); if (!new) { netdev_err(lif->netdev, "Cannot allocate queue structure\n"); @@ -414,7 +412,7 @@ static int ionic_qcq_alloc(struct ionic_lif *lif, unsigned int type, if (!new->q.info) { netdev_err(lif->netdev, "Cannot allocate queue info\n"); err = -ENOMEM; - goto err_out; + goto err_out_free_qcq; } new->q.type = type; @@ -423,7 +421,7 @@ static int ionic_qcq_alloc(struct ionic_lif *lif, unsigned int type, desc_size, sg_desc_size, pid); if (err) { netdev_err(lif->netdev, "Cannot initialize queue\n"); - goto err_out; + goto err_out_free_q_info; } if (flags & IONIC_QCQ_F_INTR) { @@ -471,46 +469,68 @@ static int ionic_qcq_alloc(struct ionic_lif *lif, unsigned int type, err = ionic_cq_init(lif, &new->cq, &new->intr, num_descs, cq_desc_size); if (err) { netdev_err(lif->netdev, "Cannot initialize completion queue\n"); - goto err_out_free_irq; + goto err_out_free_cq_info; } - new->base = dma_alloc_coherent(dev, total_size, &new->base_pa, - GFP_KERNEL); - if (!new->base) { + new->q_size = PAGE_SIZE + (num_descs * desc_size); + new->q_base = dma_alloc_coherent(dev, new->q_size, &new->q_base_pa, + GFP_KERNEL); + if (!new->q_base) { netdev_err(lif->netdev, "Cannot allocate queue DMA memory\n"); err = -ENOMEM; - goto err_out_free_irq; + goto err_out_free_cq_info; } + q_base = (void *)ALIGN((uintptr_t)new->q_base, PAGE_SIZE); + q_base_pa = ALIGN(new->q_base_pa, PAGE_SIZE); + ionic_q_map(&new->q, q_base, q_base_pa); - new->total_size = total_size; - - q_base = new->base; - q_base_pa = new->base_pa; - - cq_base = (void *)ALIGN((uintptr_t)q_base + q_size, PAGE_SIZE); - cq_base_pa = ALIGN(q_base_pa + q_size, PAGE_SIZE); + new->cq_size = PAGE_SIZE + (num_descs * cq_desc_size); + new->cq_base = dma_alloc_coherent(dev, new->cq_size, &new->cq_base_pa, + GFP_KERNEL); + if (!new->cq_base) { + netdev_err(lif->netdev, "Cannot allocate cq DMA memory\n"); + err = -ENOMEM; + goto err_out_free_q; + } + cq_base = (void *)ALIGN((uintptr_t)new->cq_base, PAGE_SIZE); + cq_base_pa = ALIGN(new->cq_base_pa, PAGE_SIZE); + ionic_cq_map(&new->cq, cq_base, cq_base_pa); + ionic_cq_bind(&new->cq, &new->q); if (flags & IONIC_QCQ_F_SG) { - sg_base = (void *)ALIGN((uintptr_t)cq_base + cq_size, - PAGE_SIZE); - sg_base_pa = ALIGN(cq_base_pa + cq_size, PAGE_SIZE); + new->sg_size = PAGE_SIZE + (num_descs * sg_desc_size); + new->sg_base = dma_alloc_coherent(dev, new->sg_size, &new->sg_base_pa, + GFP_KERNEL); + if (!new->sg_base) { + netdev_err(lif->netdev, "Cannot allocate sg DMA memory\n"); + err = -ENOMEM; + goto err_out_free_cq; + } + sg_base = (void *)ALIGN((uintptr_t)new->sg_base, PAGE_SIZE); + sg_base_pa = ALIGN(new->sg_base_pa, PAGE_SIZE); ionic_q_sg_map(&new->q, sg_base, sg_base_pa); } - ionic_q_map(&new->q, q_base, q_base_pa); - ionic_cq_map(&new->cq, cq_base, cq_base_pa); - ionic_cq_bind(&new->cq, &new->q); - *qcq = new; return 0; +err_out_free_cq: + dma_free_coherent(dev, new->cq_size, new->cq_base, new->cq_base_pa); +err_out_free_q: + dma_free_coherent(dev, new->q_size, new->q_base, new->q_base_pa); +err_out_free_cq_info: + devm_kfree(dev, new->cq.info); err_out_free_irq: if (flags & IONIC_QCQ_F_INTR) devm_free_irq(dev, new->intr.vector, &new->napi); err_out_free_intr: if (flags & IONIC_QCQ_F_INTR) ionic_intr_free(lif->ionic, new->intr.index); +err_out_free_q_info: + devm_kfree(dev, new->q.info); +err_out_free_qcq: + devm_kfree(dev, new); err_out: dev_err(dev, "qcq alloc of %s%d failed %d\n", name, index, err); return err; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.h b/drivers/net/ethernet/pensando/ionic/ionic_lif.h index 517b51190d18..aa7c1a8cbefc 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.h @@ -57,9 +57,15 @@ struct ionic_napi_stats { }; struct ionic_qcq { - void *base; - dma_addr_t base_pa; - unsigned int total_size; + void *q_base; + dma_addr_t q_base_pa; + u32 q_size; + void *cq_base; + dma_addr_t cq_base_pa; + u32 cq_size; + void *sg_base; + dma_addr_t sg_base_pa; + u32 sg_size; struct ionic_queue q; struct ionic_cq cq; struct ionic_intr_info intr; From patchwork Wed Aug 26 16:42:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shannon Nelson X-Patchwork-Id: 261890 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D797AC433DF for ; Wed, 26 Aug 2020 16:44:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A74762071E for ; Wed, 26 Aug 2020 16:44:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=pensando.io header.i=@pensando.io header.b="i5oo7zv4" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728105AbgHZQn0 (ORCPT ); Wed, 26 Aug 2020 12:43:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51180 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728036AbgHZQmd (ORCPT ); Wed, 26 Aug 2020 12:42:33 -0400 Received: from mail-pj1-x1044.google.com (mail-pj1-x1044.google.com [IPv6:2607:f8b0:4864:20::1044]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 985A4C061757 for ; Wed, 26 Aug 2020 09:42:32 -0700 (PDT) Received: by mail-pj1-x1044.google.com with SMTP id 2so1135973pjx.5 for ; Wed, 26 Aug 2020 09:42:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pensando.io; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=brxLTNsq6wnRqiuajreeSZQweUUAvKWfB6WIjFURm98=; b=i5oo7zv4OsITHWtCvvxAPJoCK/wMmFh82YBefdN0kiufJtq6n7n+osW+d44X4iBLVz QN2EWom7B9XhJEPk7xSzCpVjN1vT/1OlGw3CZtDkkZNwu0EnDs2+vpn5U/LLrDN40OTY 6KDQzAFv5cNgEXSoj5Udjs1tKTyKPb5XJc3IR8aw7YY3ME44c9MZV6iOUZ9aI/+hh+52 g562NZjI8mE9kjHpaf8YPVTXYwHAoXGopuEGEzvhcegnpRl5d7mtISpY7FZHi4OwEJr6 svUDwsU8aVvPK3P5FVLnx2/0mILMaAdRuKWKSkFMVdSAbihhUqmUPEgRfp0wjTUCamvQ tNLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=brxLTNsq6wnRqiuajreeSZQweUUAvKWfB6WIjFURm98=; b=YuRYfVDOqSfe4pdABAnF0BeDvbr8XQK3iLBform+hCKEerG3g3Ienqzsg4vF5BwXCQ xTGcg1/Qa2M5vQI5A3Rb4sFIaREXkoEkHuwjzTvk92ChDfRHBYPMSpvLuUP5x9FOGqYC zApwb16Lf1kOFlyaVHqreP1oiU1q9hBzMlUBpKLenuHptx7s7VT5jf6TCV7vXpf6ZVT+ F5QK89lpXsH3tNAHHjfIMTlb4y62LFWrS9cYIwO3zL++7YA6W/Af4SaxO3ltk5xyCKwk 81iJEeBISs+dX/o8w1ZvdevAaE9ig+dG8lAzvPBSKgDc9Eg9TKQKv9XnlFgtOrDWleBy cpZQ== X-Gm-Message-State: AOAM533rgogRjkqPyu4JWHqtw1C/OWzOALG+5rpmtnTylwLSMWE3XkjC mJrVg5mBclaG+OSm2RvaM3Dj6iocQZGOew== X-Google-Smtp-Source: ABdhPJxsaSojfIN/K1TRoL1zheZNqziRam81ewGpwvccHGyDCgFmcYX+WTgkzaIpc1SWxXJhqRvwAA== X-Received: by 2002:a17:90a:9203:: with SMTP id m3mr6813936pjo.148.1598460151193; Wed, 26 Aug 2020 09:42:31 -0700 (PDT) Received: from driver-dev1.pensando.io ([12.226.153.42]) by smtp.gmail.com with ESMTPSA id h193sm2986052pgc.42.2020.08.26.09.42.30 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Aug 2020 09:42:30 -0700 (PDT) From: Shannon Nelson To: netdev@vger.kernel.org, davem@davemloft.net Cc: Shannon Nelson Subject: [PATCH net-next 08/12] ionic: use index not pointer for queue tracking Date: Wed, 26 Aug 2020 09:42:10 -0700 Message-Id: <20200826164214.31792-9-snelson@pensando.io> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826164214.31792-1-snelson@pensando.io> References: <20200826164214.31792-1-snelson@pensando.io> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Use index counters rather than pointers for tracking head and tail in the queues to save a little memory and to perhaps slightly faster queue processing. Signed-off-by: Shannon Nelson --- .../ethernet/pensando/ionic/ionic_debugfs.c | 6 +- .../net/ethernet/pensando/ionic/ionic_dev.c | 40 ++++++----- .../net/ethernet/pensando/ionic/ionic_dev.h | 49 +++++++++---- .../net/ethernet/pensando/ionic/ionic_lif.c | 12 ++-- .../net/ethernet/pensando/ionic/ionic_lif.h | 12 ++-- .../net/ethernet/pensando/ionic/ionic_main.c | 26 ++++--- .../net/ethernet/pensando/ionic/ionic_txrx.c | 72 ++++++++++--------- 7 files changed, 129 insertions(+), 88 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_debugfs.c b/drivers/net/ethernet/pensando/ionic/ionic_debugfs.c index e12dbe4ea73d..683bbbf75115 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_debugfs.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_debugfs.c @@ -76,7 +76,7 @@ static int q_tail_show(struct seq_file *seq, void *v) { struct ionic_queue *q = seq->private; - seq_printf(seq, "%d\n", q->tail->index); + seq_printf(seq, "%d\n", q->tail_idx); return 0; } @@ -86,7 +86,7 @@ static int q_head_show(struct seq_file *seq, void *v) { struct ionic_queue *q = seq->private; - seq_printf(seq, "%d\n", q->head->index); + seq_printf(seq, "%d\n", q->head_idx); return 0; } @@ -96,7 +96,7 @@ static int cq_tail_show(struct seq_file *seq, void *v) { struct ionic_cq *cq = seq->private; - seq_printf(seq, "%d\n", cq->tail->index); + seq_printf(seq, "%d\n", cq->tail_idx); return 0; } diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.c b/drivers/net/ethernet/pensando/ionic/ionic_dev.c index 25cf376f3b40..3645673b4b18 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_dev.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.c @@ -482,7 +482,7 @@ int ionic_cq_init(struct ionic_lif *lif, struct ionic_cq *cq, cq->bound_intr = intr; cq->num_descs = num_descs; cq->desc_size = desc_size; - cq->tail = cq->info; + cq->tail_idx = 0; cq->done_color = 1; cur = cq->info; @@ -522,15 +522,18 @@ unsigned int ionic_cq_service(struct ionic_cq *cq, unsigned int work_to_do, ionic_cq_cb cb, ionic_cq_done_cb done_cb, void *done_arg) { + struct ionic_cq_info *cq_info; unsigned int work_done = 0; if (work_to_do == 0) return 0; - while (cb(cq, cq->tail)) { - if (cq->tail->last) + cq_info = &cq->info[cq->tail_idx]; + while (cb(cq, cq_info)) { + if (cq->tail_idx == cq->num_descs - 1) cq->done_color = !cq->done_color; - cq->tail = cq->tail->next; + cq->tail_idx = (cq->tail_idx + 1) & (cq->num_descs - 1); + cq_info = &cq->info[cq->tail_idx]; DEBUG_STATS_CQE_CNT(cq); if (++work_done >= work_to_do) @@ -565,8 +568,8 @@ int ionic_q_init(struct ionic_lif *lif, struct ionic_dev *idev, q->num_descs = num_descs; q->desc_size = desc_size; q->sg_desc_size = sg_desc_size; - q->tail = q->info; - q->head = q->tail; + q->tail_idx = 0; + q->head_idx = 0; q->pid = pid; snprintf(q->name, sizeof(q->name), "L%d-%s%u", lif->index, name, index); @@ -614,19 +617,22 @@ void ionic_q_post(struct ionic_queue *q, bool ring_doorbell, ionic_desc_cb cb, void *cb_arg) { struct device *dev = q->lif->ionic->dev; + struct ionic_desc_info *desc_info; struct ionic_lif *lif = q->lif; - q->head->cb = cb; - q->head->cb_arg = cb_arg; - q->head = q->head->next; + desc_info = &q->info[q->head_idx]; + desc_info->cb = cb; + desc_info->cb_arg = cb_arg; + + q->head_idx = (q->head_idx + 1) & (q->num_descs - 1); dev_dbg(dev, "lif=%d qname=%s qid=%d qtype=%d p_index=%d ringdb=%d\n", q->lif->index, q->name, q->hw_type, q->hw_index, - q->head->index, ring_doorbell); + q->head_idx, ring_doorbell); if (ring_doorbell) ionic_dbell_ring(lif->kern_dbpage, q->hw_type, - q->dbval | q->head->index); + q->dbval | q->head_idx); } static bool ionic_q_is_posted(struct ionic_queue *q, unsigned int pos) @@ -634,8 +640,8 @@ static bool ionic_q_is_posted(struct ionic_queue *q, unsigned int pos) unsigned int mask, tail, head; mask = q->num_descs - 1; - tail = q->tail->index; - head = q->head->index; + tail = q->tail_idx; + head = q->head_idx; return ((pos - tail) & mask) < ((head - tail) & mask); } @@ -648,18 +654,18 @@ void ionic_q_service(struct ionic_queue *q, struct ionic_cq_info *cq_info, void *cb_arg; /* check for empty queue */ - if (q->tail->index == q->head->index) + if (q->tail_idx == q->head_idx) return; /* stop index must be for a descriptor that is not yet completed */ if (unlikely(!ionic_q_is_posted(q, stop_index))) dev_err(q->lif->ionic->dev, "ionic stop is not posted %s stop %u tail %u head %u\n", - q->name, stop_index, q->tail->index, q->head->index); + q->name, stop_index, q->tail_idx, q->head_idx); do { - desc_info = q->tail; - q->tail = desc_info->next; + desc_info = &q->info[q->tail_idx]; + q->tail_idx = (q->tail_idx + 1) & (q->num_descs - 1); cb = desc_info->cb; cb_arg = desc_info->cb_arg; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_dev.h b/drivers/net/ethernet/pensando/ionic/ionic_dev.h index d5cba502abca..9e2ac2b8a082 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_dev.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_dev.h @@ -149,7 +149,13 @@ struct ionic_dev { }; struct ionic_cq_info { - void *cq_desc; + union { + void *cq_desc; + struct ionic_txq_comp *txcq; + struct ionic_rxq_comp *rxcq; + struct ionic_admin_comp *admincq; + struct ionic_notifyq_event *notifyq; + }; struct ionic_cq_info *next; unsigned int index; bool last; @@ -169,8 +175,17 @@ struct ionic_page_info { }; struct ionic_desc_info { - void *desc; - void *sg_desc; + union { + void *desc; + struct ionic_txq_desc *txq_desc; + struct ionic_rxq_desc *rxq_desc; + struct ionic_admin_cmd *adminq_desc; + }; + union { + void *sg_desc; + struct ionic_txq_sg_desc *txq_sg_desc; + struct ionic_rxq_sg_desc *rxq_sgl_desc; + }; struct ionic_desc_info *next; unsigned int index; unsigned int left; @@ -183,22 +198,32 @@ struct ionic_desc_info { #define IONIC_QUEUE_NAME_MAX_SZ 32 struct ionic_queue { + struct device *dev; u64 dbell_count; u64 drop; u64 stop; u64 wake; struct ionic_lif *lif; struct ionic_desc_info *info; - struct ionic_desc_info *tail; - struct ionic_desc_info *head; struct ionic_dev *idev; + u16 head_idx; + u16 tail_idx; unsigned int index; unsigned int type; unsigned int hw_index; unsigned int hw_type; u64 dbval; - void *base; - void *sg_base; + union { + void *base; + struct ionic_txq_desc *txq; + struct ionic_rxq_desc *rxq; + struct ionic_admin_cmd *adminq; + }; + union { + void *sg_base; + struct ionic_txq_sg_desc *txq_sgl; + struct ionic_rxq_sg_desc *rxq_sgl; + }; dma_addr_t base_pa; dma_addr_t sg_base_pa; unsigned int num_descs; @@ -225,9 +250,9 @@ struct ionic_cq { dma_addr_t base_pa; struct ionic_lif *lif; struct ionic_cq_info *info; - struct ionic_cq_info *tail; struct ionic_queue *bound_q; struct ionic_intr_info *bound_intr; + u16 tail_idx; bool done_color; unsigned int num_descs; u64 compl_count; @@ -246,12 +271,12 @@ static inline void ionic_intr_init(struct ionic_dev *idev, static inline unsigned int ionic_q_space_avail(struct ionic_queue *q) { - unsigned int avail = q->tail->index; + unsigned int avail = q->tail_idx; - if (q->head->index >= avail) - avail += q->head->left - 1; + if (q->head_idx >= avail) + avail += q->num_descs - q->head_idx - 1; else - avail -= q->head->index + 1; + avail -= q->head_idx + 1; return avail; } diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index 016d55ad1f6a..b77827e9355c 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -632,9 +632,9 @@ static int ionic_lif_txq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) dev_dbg(dev, "txq_init.ver %d\n", ctx.cmd.q_init.ver); dev_dbg(dev, "txq_init.intr_index %d\n", ctx.cmd.q_init.intr_index); - q->tail = q->info; - q->head = q->tail; - cq->tail = cq->info; + q->tail_idx = 0; + q->head_idx = 0; + cq->tail_idx = 0; err = ionic_adminq_post_wait(lif, &ctx); if (err) @@ -689,9 +689,9 @@ static int ionic_lif_rxq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) dev_dbg(dev, "rxq_init.ver %d\n", ctx.cmd.q_init.ver); dev_dbg(dev, "rxq_init.intr_index %d\n", ctx.cmd.q_init.intr_index); - q->tail = q->info; - q->head = q->tail; - cq->tail = cq->info; + q->tail_idx = 0; + q->head_idx = 0; + cq->tail_idx = 0; err = ionic_adminq_post_wait(lif, &ctx); if (err) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.h b/drivers/net/ethernet/pensando/ionic/ionic_lif.h index aa7c1a8cbefc..b8f774b1db3a 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.h @@ -244,14 +244,15 @@ int ionic_lif_rss_config(struct ionic_lif *lif, u16 types, int ionic_reset_queues(struct ionic_lif *lif, ionic_reset_cb cb, void *arg); -static inline void debug_stats_txq_post(struct ionic_queue *q, - struct ionic_txq_desc *desc, bool dbell) +static inline void debug_stats_txq_post(struct ionic_queue *q, bool dbell) { - u8 num_sg_elems = ((le64_to_cpu(desc->cmd) >> IONIC_TXQ_DESC_NSGE_SHIFT) - & IONIC_TXQ_DESC_NSGE_MASK); + struct ionic_txq_desc *desc = &q->txq[q->head_idx]; + u8 num_sg_elems; q->dbell_count += dbell; + num_sg_elems = ((le64_to_cpu(desc->cmd) >> IONIC_TXQ_DESC_NSGE_SHIFT) + & IONIC_TXQ_DESC_NSGE_MASK); if (num_sg_elems > (IONIC_MAX_NUM_SG_CNTR - 1)) num_sg_elems = IONIC_MAX_NUM_SG_CNTR - 1; @@ -272,8 +273,7 @@ static inline void debug_stats_napi_poll(struct ionic_qcq *qcq, #define DEBUG_STATS_CQE_CNT(cq) ((cq)->compl_count++) #define DEBUG_STATS_RX_BUFF_CNT(q) ((q)->lif->rxqstats[q->index].buffers_posted++) #define DEBUG_STATS_INTR_REARM(intr) ((intr)->rearm_count++) -#define DEBUG_STATS_TXQ_POST(q, txdesc, dbell) \ - debug_stats_txq_post(q, txdesc, dbell) +#define DEBUG_STATS_TXQ_POST(q, dbell) debug_stats_txq_post(q, dbell) #define DEBUG_STATS_NAPI_POLL(qcq, work_done) \ debug_stats_napi_poll(qcq, work_done) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_main.c b/drivers/net/ethernet/pensando/ionic/ionic_main.c index df5b9bcc3aba..2b72a51be1d0 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_main.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_main.c @@ -181,15 +181,17 @@ static const char *ionic_opcode_to_str(enum ionic_cmd_opcode opcode) static void ionic_adminq_flush(struct ionic_lif *lif) { - struct ionic_queue *adminq = &lif->adminqcq->q; + struct ionic_queue *q = &lif->adminqcq->q; + struct ionic_desc_info *desc_info; spin_lock(&lif->adminq_lock); - while (adminq->tail != adminq->head) { - memset(adminq->tail->desc, 0, sizeof(union ionic_adminq_cmd)); - adminq->tail->cb = NULL; - adminq->tail->cb_arg = NULL; - adminq->tail = adminq->tail->next; + while (q->tail_idx != q->head_idx) { + desc_info = &q->info[q->tail_idx]; + memset(desc_info->desc, 0, sizeof(union ionic_adminq_cmd)); + desc_info->cb = NULL; + desc_info->cb_arg = NULL; + q->tail_idx = (q->tail_idx + 1) & (q->num_descs - 1); } spin_unlock(&lif->adminq_lock); } @@ -245,7 +247,8 @@ static void ionic_adminq_cb(struct ionic_queue *q, static int ionic_adminq_post(struct ionic_lif *lif, struct ionic_admin_ctx *ctx) { - struct ionic_queue *adminq; + struct ionic_desc_info *desc_info; + struct ionic_queue *q; int err = 0; WARN_ON(in_interrupt()); @@ -253,10 +256,10 @@ static int ionic_adminq_post(struct ionic_lif *lif, struct ionic_admin_ctx *ctx) if (!lif->adminqcq) return -EIO; - adminq = &lif->adminqcq->q; + q = &lif->adminqcq->q; spin_lock(&lif->adminq_lock); - if (!ionic_q_has_space(adminq, 1)) { + if (!ionic_q_has_space(q, 1)) { err = -ENOSPC; goto err_out; } @@ -265,13 +268,14 @@ static int ionic_adminq_post(struct ionic_lif *lif, struct ionic_admin_ctx *ctx) if (err) goto err_out; - memcpy(adminq->head->desc, &ctx->cmd, sizeof(ctx->cmd)); + desc_info = &q->info[q->head_idx]; + memcpy(desc_info->desc, &ctx->cmd, sizeof(ctx->cmd)); dev_dbg(&lif->netdev->dev, "post admin queue command:\n"); dynamic_hex_dump("cmd ", DUMP_PREFIX_OFFSET, 16, 1, &ctx->cmd, sizeof(ctx->cmd), true); - ionic_q_post(adminq, true, ionic_adminq_cb, ctx); + ionic_q_post(q, true, ionic_adminq_cb, ctx); err_out: spin_unlock(&lif->adminq_lock); diff --git a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c index 220e132164e2..c3291decd4c3 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_txrx.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_txrx.c @@ -22,7 +22,7 @@ static bool ionic_tx_service(struct ionic_cq *cq, struct ionic_cq_info *cq_info) static inline void ionic_txq_post(struct ionic_queue *q, bool ring_dbell, ionic_desc_cb cb_func, void *cb_arg) { - DEBUG_STATS_TXQ_POST(q, q->head->desc, ring_dbell); + DEBUG_STATS_TXQ_POST(q, ring_dbell); ionic_q_post(q, ring_dbell, cb_func, cb_arg); } @@ -235,14 +235,14 @@ static bool ionic_rx_service(struct ionic_cq *cq, struct ionic_cq_info *cq_info) return false; /* check for empty queue */ - if (q->tail->index == q->head->index) + if (q->tail_idx == q->head_idx) return false; - desc_info = q->tail; + desc_info = &q->info[q->tail_idx]; if (desc_info->index != le16_to_cpu(comp->comp_index)) return false; - q->tail = desc_info->next; + q->tail_idx = (q->tail_idx + 1) & (q->num_descs - 1); /* clean the related q entry, only one per qc completion */ ionic_rx_clean(q, desc_info, cq_info, desc_info->cb_arg); @@ -338,7 +338,7 @@ void ionic_rx_fill(struct ionic_queue *q) for (i = ionic_q_space_avail(q); i; i--) { remain_len = len; - desc_info = q->head; + desc_info = &q->info[q->head_idx]; desc = desc_info->desc; sg_desc = desc_info->sg_desc; page_info = &desc_info->pages[0]; @@ -387,7 +387,7 @@ void ionic_rx_fill(struct ionic_queue *q) } ionic_dbell_ring(q->lif->kern_dbpage, q->hw_type, - q->dbval | q->head->index); + q->dbval | q->head_idx); } static void ionic_rx_fill_cb(void *arg) @@ -397,25 +397,29 @@ static void ionic_rx_fill_cb(void *arg) void ionic_rx_empty(struct ionic_queue *q) { - struct ionic_desc_info *cur; + struct ionic_desc_info *desc_info; struct ionic_rxq_desc *desc; unsigned int i; + u16 idx; - for (cur = q->tail; cur != q->head; cur = cur->next) { - desc = cur->desc; + idx = q->tail_idx; + while (idx != q->head_idx) { + desc_info = &q->info[idx]; + desc = desc_info->desc; desc->addr = 0; desc->len = 0; - for (i = 0; i < cur->npages; i++) { - if (likely(cur->pages[i].page)) { - ionic_rx_page_free(q, cur->pages[i].page, - cur->pages[i].dma_addr); - cur->pages[i].page = NULL; - cur->pages[i].dma_addr = 0; + for (i = 0; i < desc_info->npages; i++) { + if (likely(desc_info->pages[i].page)) { + ionic_rx_page_free(q, desc_info->pages[i].page, + desc_info->pages[i].dma_addr); + desc_info->pages[i].page = NULL; + desc_info->pages[i].dma_addr = 0; } } - cur->cb_arg = NULL; + desc_info->cb_arg = NULL; + idx = (idx + 1) & (q->num_descs - 1); } } @@ -630,9 +634,9 @@ static bool ionic_tx_service(struct ionic_cq *cq, struct ionic_cq_info *cq_info) * several q entries completed for each cq completion */ do { - desc_info = q->tail; - q->tail = desc_info->next; - ionic_tx_clean(q, desc_info, cq->tail, desc_info->cb_arg); + desc_info = &q->info[q->tail_idx]; + q->tail_idx = (q->tail_idx + 1) & (q->num_descs - 1); + ionic_tx_clean(q, desc_info, cq_info, desc_info->cb_arg); desc_info->cb = NULL; desc_info->cb_arg = NULL; } while (desc_info->index != le16_to_cpu(comp->comp_index)); @@ -658,9 +662,9 @@ void ionic_tx_empty(struct ionic_queue *q) int done = 0; /* walk the not completed tx entries, if any */ - while (q->head != q->tail) { - desc_info = q->tail; - q->tail = desc_info->next; + while (q->head_idx != q->tail_idx) { + desc_info = &q->info[q->tail_idx]; + q->tail_idx = (q->tail_idx + 1) & (q->num_descs - 1); ionic_tx_clean(q, desc_info, NULL, desc_info->cb_arg); desc_info->cb = NULL; desc_info->cb_arg = NULL; @@ -748,8 +752,8 @@ static void ionic_tx_tso_post(struct ionic_queue *q, struct ionic_txq_desc *desc static struct ionic_txq_desc *ionic_tx_tso_next(struct ionic_queue *q, struct ionic_txq_sg_elem **elem) { - struct ionic_txq_sg_desc *sg_desc = q->head->sg_desc; - struct ionic_txq_desc *desc = q->head->desc; + struct ionic_txq_sg_desc *sg_desc = q->info[q->head_idx].txq_sg_desc; + struct ionic_txq_desc *desc = q->info[q->head_idx].txq_desc; *elem = sg_desc->elems; return desc; @@ -758,13 +762,13 @@ static struct ionic_txq_desc *ionic_tx_tso_next(struct ionic_queue *q, static int ionic_tx_tso(struct ionic_queue *q, struct sk_buff *skb) { struct ionic_tx_stats *stats = q_to_tx_stats(q); - struct ionic_desc_info *abort = q->head; + struct ionic_desc_info *rewind_desc_info; struct device *dev = q->lif->ionic->dev; - struct ionic_desc_info *rewind = abort; struct ionic_txq_sg_elem *elem; struct ionic_txq_desc *desc; unsigned int frag_left = 0; unsigned int offset = 0; + u16 abort = q->head_idx; unsigned int len_left; dma_addr_t desc_addr; unsigned int hdrlen; @@ -772,6 +776,7 @@ static int ionic_tx_tso(struct ionic_queue *q, struct sk_buff *skb) unsigned int seglen; u64 total_bytes = 0; u64 total_pkts = 0; + u16 rewind = abort; unsigned int left; unsigned int len; unsigned int mss; @@ -916,19 +921,20 @@ static int ionic_tx_tso(struct ionic_queue *q, struct sk_buff *skb) return 0; err_out_abort: - while (rewind->desc != q->head->desc) { - ionic_tx_clean(q, rewind, NULL, NULL); - rewind = rewind->next; + while (rewind != q->head_idx) { + rewind_desc_info = &q->info[rewind]; + ionic_tx_clean(q, rewind_desc_info, NULL, NULL); + rewind = (rewind + 1) & (q->num_descs - 1); } - q->head = abort; + q->head_idx = abort; return -ENOMEM; } static int ionic_tx_calc_csum(struct ionic_queue *q, struct sk_buff *skb) { + struct ionic_txq_desc *desc = q->info[q->head_idx].txq_desc; struct ionic_tx_stats *stats = q_to_tx_stats(q); - struct ionic_txq_desc *desc = q->head->desc; struct device *dev = q->lif->ionic->dev; dma_addr_t dma_addr; bool has_vlan; @@ -967,8 +973,8 @@ static int ionic_tx_calc_csum(struct ionic_queue *q, struct sk_buff *skb) static int ionic_tx_calc_no_csum(struct ionic_queue *q, struct sk_buff *skb) { + struct ionic_txq_desc *desc = q->info[q->head_idx].txq_desc; struct ionic_tx_stats *stats = q_to_tx_stats(q); - struct ionic_txq_desc *desc = q->head->desc; struct device *dev = q->lif->ionic->dev; dma_addr_t dma_addr; bool has_vlan; @@ -1002,7 +1008,7 @@ static int ionic_tx_calc_no_csum(struct ionic_queue *q, struct sk_buff *skb) static int ionic_tx_skb_frags(struct ionic_queue *q, struct sk_buff *skb) { - struct ionic_txq_sg_desc *sg_desc = q->head->sg_desc; + struct ionic_txq_sg_desc *sg_desc = q->info[q->head_idx].txq_sg_desc; unsigned int len_left = skb->len - skb_headlen(skb); struct ionic_txq_sg_elem *elem = sg_desc->elems; struct ionic_tx_stats *stats = q_to_tx_stats(q); From patchwork Wed Aug 26 16:42:12 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shannon Nelson X-Patchwork-Id: 261891 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EC5AC433E1 for ; Wed, 26 Aug 2020 16:43:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C9150206FA for ; Wed, 26 Aug 2020 16:43:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=pensando.io header.i=@pensando.io header.b="phGoGTcp" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728141AbgHZQnp (ORCPT ); Wed, 26 Aug 2020 12:43:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51188 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728039AbgHZQmf (ORCPT ); Wed, 26 Aug 2020 12:42:35 -0400 Received: from mail-pf1-x443.google.com (mail-pf1-x443.google.com [IPv6:2607:f8b0:4864:20::443]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6C187C061796 for ; Wed, 26 Aug 2020 09:42:34 -0700 (PDT) Received: by mail-pf1-x443.google.com with SMTP id k18so1260564pfp.7 for ; Wed, 26 Aug 2020 09:42:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pensando.io; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=5avBMlEduRdsosvyiPhQlDyXGD+4H+ubevkBKjfZsr4=; b=phGoGTcp5RaQm++XjZp8D+63O/o4NLEvxJe5kOa6eLZWp5WVQR73ycccAmyNLwkZwq N/jT8e+cv9Z5pQ/rAGRnHDApgP7pUSo8UF+JoSHbRsZX7RqVZgn2mUtvIsJ7M5ri1MC1 wuu0WyBnGcI9/+tEcrqlUb9bsmwj/EWlwlg/e5nHImWKrfSFPNJxHZP0uxugtNW/6zDi rZcVWh5Z3dbB4XaAnpCKGmKoTpiLpY4SApF/Dfo/e3Ev3Mz3VKTV0Q+dkOJG/8666Qx0 7rvmkXCooAzCKCIv/U3wc2Q4WF87kOKT5lQvRdhtzVfbcRskWWbA1gmkkgUXWYsEVeMe /6Cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=5avBMlEduRdsosvyiPhQlDyXGD+4H+ubevkBKjfZsr4=; b=ktRlW1y17VnC+f9WgPRKE8DA5i6lq5ZPwOWao6cdHEejADCA6i3OBeO17TYLXqzDPz /UAODXBmQpHNIz3djonBzayjByfbu1mSdL8yBlQ4f9D2ttP9A6B2kfHlZeSHETZ+N/mL VoyqE+TF18ahKC5IM+IhdLnzhDMT85tiXr9em98BVPxHHxkvKY9QjMbDr2dXIyGw1jG+ eG0tRQa7bV5xToqfXQhMbS0efaat4ia6aFl4XbycTo56RsP15JUim18abMPx0N7fDj5G vg2GHWmXCLD5Azd31TW8BXh3S7s9BCtSM7QCw/gdwg1kRLCn8569ya/W0Voc5aM/WjfE biog== X-Gm-Message-State: AOAM5321H40hIZKnL1l7B9eIhQa1Hd3EqURRRcmIa1jNL1Zo/lAk8N2c OmwhEfkgzPfIxQcwrTSzvF3aCnR9K4k+ng== X-Google-Smtp-Source: ABdhPJwyc37B2lkEZwGxxP1OUYeFyixwm2lHD6LGw6RTr4RaRxKm0NKYnGlYP/E/n7n2X40cCtftdw== X-Received: by 2002:a62:2704:: with SMTP id n4mr13233386pfn.246.1598460153369; Wed, 26 Aug 2020 09:42:33 -0700 (PDT) Received: from driver-dev1.pensando.io ([12.226.153.42]) by smtp.gmail.com with ESMTPSA id h193sm2986052pgc.42.2020.08.26.09.42.32 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Aug 2020 09:42:32 -0700 (PDT) From: Shannon Nelson To: netdev@vger.kernel.org, davem@davemloft.net Cc: Shannon Nelson Subject: [PATCH net-next 10/12] ionic: change the descriptor ring length without full reset Date: Wed, 26 Aug 2020 09:42:12 -0700 Message-Id: <20200826164214.31792-11-snelson@pensando.io> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826164214.31792-1-snelson@pensando.io> References: <20200826164214.31792-1-snelson@pensando.io> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The original way of changing ring length was to completely tear down the lif's queue structure and then rebuild it, while running the risk of allocations that might fail in the middle and leave us with a broken driver. Instead, we can set up all the new queue and descriptor allocations first, then swap them out and delete the old allocations. If the new allocations fail, we report the error, stay with the old setup and continue running. This gives us a safer path, and a smaller window of time where we're not processing traffic. Signed-off-by: Shannon Nelson --- .../ethernet/pensando/ionic/ionic_ethtool.c | 35 +++- .../net/ethernet/pensando/ionic/ionic_lif.c | 161 +++++++++++++++++- .../net/ethernet/pensando/ionic/ionic_lif.h | 19 ++- 3 files changed, 201 insertions(+), 14 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c index 73af0ecc5495..adaefa5fe883 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c @@ -493,18 +493,14 @@ static void ionic_get_ringparam(struct net_device *netdev, ring->rx_pending = lif->nrxq_descs; } -static void ionic_set_ringsize(struct ionic_lif *lif, void *arg) -{ - struct ethtool_ringparam *ring = arg; - - lif->ntxq_descs = ring->tx_pending; - lif->nrxq_descs = ring->rx_pending; -} - static int ionic_set_ringparam(struct net_device *netdev, struct ethtool_ringparam *ring) { struct ionic_lif *lif = netdev_priv(netdev); + struct ionic_queue_params qparam; + int err; + + ionic_init_queue_params(lif, &qparam); if (ring->rx_mini_pending || ring->rx_jumbo_pending) { netdev_info(netdev, "Changing jumbo or mini descriptors not supported\n"); @@ -522,7 +518,28 @@ static int ionic_set_ringparam(struct net_device *netdev, ring->rx_pending == lif->nrxq_descs) return 0; - return ionic_reset_queues(lif, ionic_set_ringsize, ring); + if (ring->tx_pending != lif->ntxq_descs) + netdev_info(netdev, "Changing Tx ring size from %d to %d\n", + lif->ntxq_descs, ring->tx_pending); + + if (ring->rx_pending != lif->nrxq_descs) + netdev_info(netdev, "Changing Rx ring size from %d to %d\n", + lif->nrxq_descs, ring->rx_pending); + + /* if we're not running, just set the values and return */ + if (!netif_running(lif->netdev)) { + lif->ntxq_descs = ring->tx_pending; + lif->nrxq_descs = ring->rx_pending; + return 0; + } + + qparam.ntxq_descs = ring->tx_pending; + qparam.nrxq_descs = ring->rx_pending; + err = ionic_reconfigure_queues(lif, &qparam); + if (err) + netdev_info(netdev, "Ring reconfiguration failed, changes canceled: %d\n", err); + + return err; } static void ionic_get_channels(struct net_device *netdev, diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index d82ae717bc6c..40e249ca1ea3 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -333,10 +333,14 @@ static void ionic_qcq_free(struct ionic_lif *lif, struct ionic_qcq *qcq) ionic_intr_free(lif->ionic, qcq->intr.index); } - devm_kfree(dev, qcq->cq.info); - qcq->cq.info = NULL; - devm_kfree(dev, qcq->q.info); - qcq->q.info = NULL; + if (qcq->cq.info) { + devm_kfree(dev, qcq->cq.info); + qcq->cq.info = NULL; + } + if (qcq->q.info) { + devm_kfree(dev, qcq->q.info); + qcq->q.info = NULL; + } devm_kfree(dev, qcq); } @@ -2037,6 +2041,155 @@ static const struct net_device_ops ionic_netdev_ops = { .ndo_get_vf_stats = ionic_get_vf_stats, }; +static void ionic_swap_queues(struct ionic_qcq *a, struct ionic_qcq *b) +{ + /* only swapping the queues, not the napi, flags, or other stuff */ + swap(a->q.num_descs, b->q.num_descs); + swap(a->q.base, b->q.base); + swap(a->q.base_pa, b->q.base_pa); + swap(a->q.info, b->q.info); + swap(a->q_base, b->q_base); + swap(a->q_base_pa, b->q_base_pa); + swap(a->q_size, b->q_size); + + swap(a->q.sg_base, b->q.sg_base); + swap(a->q.sg_base_pa, b->q.sg_base_pa); + swap(a->sg_base, b->sg_base); + swap(a->sg_base_pa, b->sg_base_pa); + swap(a->sg_size, b->sg_size); + + swap(a->cq.num_descs, b->cq.num_descs); + swap(a->cq.base, b->cq.base); + swap(a->cq.base_pa, b->cq.base_pa); + swap(a->cq.info, b->cq.info); + swap(a->cq_base, b->cq_base); + swap(a->cq_base_pa, b->cq_base_pa); + swap(a->cq_size, b->cq_size); +} + +int ionic_reconfigure_queues(struct ionic_lif *lif, + struct ionic_queue_params *qparam) +{ + struct ionic_qcq **tx_qcqs = NULL; + struct ionic_qcq **rx_qcqs = NULL; + unsigned int sg_desc_sz; + unsigned int flags; + int err = -ENOMEM; + unsigned int i; + + /* allocate temporary qcq arrays to hold new queue structs */ + if (qparam->ntxq_descs != lif->ntxq_descs) { + tx_qcqs = devm_kcalloc(lif->ionic->dev, lif->nxqs, + sizeof(struct ionic_qcq *), GFP_KERNEL); + if (!tx_qcqs) + goto err_out; + } + if (qparam->nrxq_descs != lif->nrxq_descs) { + rx_qcqs = devm_kcalloc(lif->ionic->dev, lif->nxqs, + sizeof(struct ionic_qcq *), GFP_KERNEL); + if (!rx_qcqs) + goto err_out; + } + + /* allocate new desc_info and rings with no interrupt flag */ + if (lif->qtype_info[IONIC_QTYPE_TXQ].version >= 1 && + lif->qtype_info[IONIC_QTYPE_TXQ].sg_desc_sz == + sizeof(struct ionic_txq_sg_desc_v1)) + sg_desc_sz = sizeof(struct ionic_txq_sg_desc_v1); + else + sg_desc_sz = sizeof(struct ionic_txq_sg_desc); + + if (tx_qcqs) { + for (i = 0; i < lif->nxqs; i++) { + flags = lif->txqcqs[i]->flags & ~IONIC_QCQ_F_INTR; + err = ionic_qcq_alloc(lif, IONIC_QTYPE_TXQ, i, "tx", flags, + qparam->ntxq_descs, + sizeof(struct ionic_txq_desc), + sizeof(struct ionic_txq_comp), + sg_desc_sz, + lif->kern_pid, &tx_qcqs[i]); + if (err) + goto err_out; + } + } + + if (rx_qcqs) { + for (i = 0; i < lif->nxqs; i++) { + flags = lif->rxqcqs[i]->flags & ~IONIC_QCQ_F_INTR; + err = ionic_qcq_alloc(lif, IONIC_QTYPE_RXQ, i, "rx", flags, + qparam->nrxq_descs, + sizeof(struct ionic_rxq_desc), + sizeof(struct ionic_rxq_comp), + sizeof(struct ionic_rxq_sg_desc), + lif->kern_pid, &rx_qcqs[i]); + if (err) + goto err_out; + } + } + + /* stop and clean the queues */ + mutex_lock(&lif->queue_lock); + netif_device_detach(lif->netdev); + ionic_stop_queues(lif); + ionic_txrx_deinit(lif); + + /* swap new desc_info and rings, keeping existing interrupt config */ + if (tx_qcqs) { + lif->ntxq_descs = qparam->ntxq_descs; + for (i = 0; i < lif->nxqs; i++) + ionic_swap_queues(lif->txqcqs[i], tx_qcqs[i]); + } + + if (rx_qcqs) { + lif->nrxq_descs = qparam->nrxq_descs; + for (i = 0; i < lif->nxqs; i++) + ionic_swap_queues(lif->rxqcqs[i], rx_qcqs[i]); + } + + /* re-init the queues */ + err = ionic_txrx_init(lif); + if (err) + goto err_out_attach_unlock; + + /* don't start the queues unless we have link */ + if (netif_carrier_ok(lif->netdev)) { + err = ionic_start_queues(lif); + if (err) + goto err_out_attach_unlock; + } + +err_out_attach_unlock: + netif_device_attach(lif->netdev); + mutex_unlock(&lif->queue_lock); + +err_out: + /* free old allocs without cleaning intr */ + for (i = 0; i < lif->nxqs; i++) { + if (tx_qcqs && tx_qcqs[i]) { + tx_qcqs[i]->flags &= ~IONIC_QCQ_F_INTR; + ionic_qcq_free(lif, tx_qcqs[i]); + tx_qcqs[i] = NULL; + } + if (rx_qcqs && rx_qcqs[i]) { + rx_qcqs[i]->flags &= ~IONIC_QCQ_F_INTR; + ionic_qcq_free(lif, rx_qcqs[i]); + rx_qcqs[i] = NULL; + } + } + + /* free q array */ + if (rx_qcqs) { + devm_kfree(lif->ionic->dev, rx_qcqs); + rx_qcqs = NULL; + } + if (tx_qcqs) { + devm_kfree(lif->ionic->dev, tx_qcqs); + tx_qcqs = NULL; + } + + return err; +} + int ionic_reset_queues(struct ionic_lif *lif, ionic_reset_cb cb, void *arg) { bool running; diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.h b/drivers/net/ethernet/pensando/ionic/ionic_lif.h index b8f774b1db3a..1df3e1e4107b 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.h +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.h @@ -207,6 +207,22 @@ struct ionic_lif { struct work_struct tx_timeout_work; }; +struct ionic_queue_params { + unsigned int nxqs; + unsigned int ntxq_descs; + unsigned int nrxq_descs; + unsigned int intr_split; +}; + +static inline void ionic_init_queue_params(struct ionic_lif *lif, + struct ionic_queue_params *qparam) +{ + qparam->nxqs = lif->nxqs; + qparam->ntxq_descs = lif->ntxq_descs; + qparam->nrxq_descs = lif->nrxq_descs; + qparam->intr_split = test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state); +} + static inline u32 ionic_coal_usec_to_hw(struct ionic *ionic, u32 usecs) { u32 mult = le32_to_cpu(ionic->ident.dev.intr_coal_mult); @@ -241,7 +257,8 @@ int ionic_lif_identify(struct ionic *ionic, u8 lif_type, int ionic_lif_size(struct ionic *ionic); int ionic_lif_rss_config(struct ionic_lif *lif, u16 types, const u8 *key, const u32 *indir); - +int ionic_reconfigure_queues(struct ionic_lif *lif, + struct ionic_queue_params *qparam); int ionic_reset_queues(struct ionic_lif *lif, ionic_reset_cb cb, void *arg); static inline void debug_stats_txq_post(struct ionic_queue *q, bool dbell) From patchwork Wed Aug 26 16:42:13 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Shannon Nelson X-Patchwork-Id: 261889 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E43D7C433DF for ; Wed, 26 Aug 2020 16:44:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A7B13206FA for ; Wed, 26 Aug 2020 16:44:17 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=pensando.io header.i=@pensando.io header.b="221LoHeG" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728153AbgHZQoJ (ORCPT ); Wed, 26 Aug 2020 12:44:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51196 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727824AbgHZQmn (ORCPT ); Wed, 26 Aug 2020 12:42:43 -0400 Received: from mail-pj1-x1043.google.com (mail-pj1-x1043.google.com [IPv6:2607:f8b0:4864:20::1043]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4A37CC0613ED for ; Wed, 26 Aug 2020 09:42:38 -0700 (PDT) Received: by mail-pj1-x1043.google.com with SMTP id ds1so1145340pjb.1 for ; Wed, 26 Aug 2020 09:42:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pensando.io; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=a0kLxXw1SKsTm4WmwqdV5WcFKTqHNh8VnG7umPeF4yY=; b=221LoHeGT8TXQImAAhiTelq0T50UjxhUiw+Y+Vsd068k0tY0A2yA9SEZmxAnm+qZGK 2Ikw5+M88INl2ipLTUIGE4Z9Gh0hc4mRc8tWFAttkuwv9LFtyuD5ECxOAkNGHmUV9KeE TqQEkP098yLoPYZD2zszj1H8F4PfcB3CKXWrMWeMryaairMHAOXFOh2zJt7ZZpQsHK69 5FNQ9MEbsGwVBhorZ0P6dCes3h6M83KBeCVcrgPQefuJer+f19nkqsepLTxU4LlJaAFb 8YFHvqh1YavPxg/nEYF1JVwhIpVZhGbh/vd2zlmrPac3mg5hNqzoQVdjI4ggKKrbL+0P Bxug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=a0kLxXw1SKsTm4WmwqdV5WcFKTqHNh8VnG7umPeF4yY=; b=JVkKp68MV9BwdzyFLCbZCBUJp4NTEzj8Yaiwp6ej4tB1ybu0hFL0GRCptASIp7Ku1u sI2kLkIvP3dl1K0midR3OW416Eicr3FL93H2+uSZy8zJ/EtnFRYd8E7hD5kKzMgFRi9D gIv5fFUrc9ALgfLc36EGrdt+vaAW21gZcaxn6gS3LtYzKCebvvM3GuXDMyWAYpQrfWQV ne2SSAoax2J/hfqHF6Z/zn8Z6mdD+uoUIPPtUxccO220Mjd5JPS56kJ1gGm+E4WH98vk PVB99w1MoKDE1oXvZHCZe1VOqFtA1oiekG2G4NiG/jWiRLSA+wPJ6Pa+KLX3CVwdFcKK 2ZzQ== X-Gm-Message-State: AOAM532PYhFe4n+AWTFv2XCdXgcydg4d9fMudajXJwknNV53nvTjhU5Y cRtoz84flHRhHEnslNMXv5DG3LqakZZtMw== X-Google-Smtp-Source: ABdhPJz38gNXDVywAy/CQf9x/jmyOhuvMTTR2Uwib26YWJrucXhQ2c+JrlTM8Fya1fwRtKJIqaXxGg== X-Received: by 2002:a17:90a:f28a:: with SMTP id fs10mr6551436pjb.219.1598460157053; Wed, 26 Aug 2020 09:42:37 -0700 (PDT) Received: from driver-dev1.pensando.io ([12.226.153.42]) by smtp.gmail.com with ESMTPSA id h193sm2986052pgc.42.2020.08.26.09.42.33 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 26 Aug 2020 09:42:34 -0700 (PDT) From: Shannon Nelson To: netdev@vger.kernel.org, davem@davemloft.net Cc: Shannon Nelson Subject: [PATCH net-next 11/12] ionic: change queue count with no reset Date: Wed, 26 Aug 2020 09:42:13 -0700 Message-Id: <20200826164214.31792-12-snelson@pensando.io> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200826164214.31792-1-snelson@pensando.io> References: <20200826164214.31792-1-snelson@pensando.io> Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add to our new ionic_reconfigure_queues() to also be able to change the number of queues in use, and to change the queue interrupt layout between split and combined. Signed-off-by: Shannon Nelson --- .../ethernet/pensando/ionic/ionic_ethtool.c | 88 +++++--- .../net/ethernet/pensando/ionic/ionic_lif.c | 206 +++++++++++++----- 2 files changed, 205 insertions(+), 89 deletions(-) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c index adaefa5fe883..00aad7168915 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_ethtool.c @@ -561,32 +561,15 @@ static void ionic_get_channels(struct net_device *netdev, } } -static void ionic_set_queuecount(struct ionic_lif *lif, void *arg) -{ - struct ethtool_channels *ch = arg; - - if (ch->combined_count) { - lif->nxqs = ch->combined_count; - if (test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) { - clear_bit(IONIC_LIF_F_SPLIT_INTR, lif->state); - lif->tx_coalesce_usecs = lif->rx_coalesce_usecs; - lif->tx_coalesce_hw = lif->rx_coalesce_hw; - netdev_info(lif->netdev, "Sharing queue interrupts\n"); - } - } else { - lif->nxqs = ch->rx_count; - if (!test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) { - set_bit(IONIC_LIF_F_SPLIT_INTR, lif->state); - netdev_info(lif->netdev, "Splitting queue interrupts\n"); - } - } -} - static int ionic_set_channels(struct net_device *netdev, struct ethtool_channels *ch) { struct ionic_lif *lif = netdev_priv(netdev); - int new_cnt; + struct ionic_queue_params qparam; + int max_cnt; + int err; + + ionic_init_queue_params(lif, &qparam); if (ch->rx_count != ch->tx_count) { netdev_info(netdev, "The rx and tx count must be equal\n"); @@ -594,20 +577,63 @@ static int ionic_set_channels(struct net_device *netdev, } if (ch->combined_count && ch->rx_count) { - netdev_info(netdev, "Use either combined_count or rx/tx_count, not both\n"); + netdev_info(netdev, "Use either combined or rx and tx, not both\n"); return -EINVAL; } - if (ch->combined_count) - new_cnt = ch->combined_count; - else - new_cnt = ch->rx_count; + max_cnt = lif->ionic->ntxqs_per_lif; + if (ch->combined_count) { + if (ch->combined_count > max_cnt) + return -EINVAL; + + if (test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) + netdev_info(lif->netdev, "Sharing queue interrupts\n"); + else if (ch->combined_count == lif->nxqs) + return 0; - if (lif->nxqs != new_cnt) - netdev_info(netdev, "Changing queue count from %d to %d\n", - lif->nxqs, new_cnt); + if (lif->nxqs != ch->combined_count) + netdev_info(netdev, "Changing queue count from %d to %d\n", + lif->nxqs, ch->combined_count); - return ionic_reset_queues(lif, ionic_set_queuecount, ch); + qparam.nxqs = ch->combined_count; + qparam.intr_split = 0; + } else { + max_cnt /= 2; + if (ch->rx_count > max_cnt) + return -EINVAL; + + if (!test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) + netdev_info(lif->netdev, "Splitting queue interrupts\n"); + else if (ch->rx_count == lif->nxqs) + return 0; + + if (lif->nxqs != ch->rx_count) + netdev_info(netdev, "Changing queue count from %d to %d\n", + lif->nxqs, ch->rx_count); + + qparam.nxqs = ch->rx_count; + qparam.intr_split = 1; + } + + /* if we're not running, just set the values and return */ + if (!netif_running(lif->netdev)) { + lif->nxqs = qparam.nxqs; + + if (qparam.intr_split) { + set_bit(IONIC_LIF_F_SPLIT_INTR, lif->state); + } else { + clear_bit(IONIC_LIF_F_SPLIT_INTR, lif->state); + lif->tx_coalesce_usecs = lif->rx_coalesce_usecs; + lif->tx_coalesce_hw = lif->rx_coalesce_hw; + } + return 0; + } + + err = ionic_reconfigure_queues(lif, &qparam); + if (err) + netdev_info(netdev, "Queue reconfiguration failed, changes canceled: %d\n", err); + + return err; } static u32 ionic_get_priv_flags(struct net_device *netdev) diff --git a/drivers/net/ethernet/pensando/ionic/ionic_lif.c b/drivers/net/ethernet/pensando/ionic/ionic_lif.c index 40e249ca1ea3..0da975a45692 100644 --- a/drivers/net/ethernet/pensando/ionic/ionic_lif.c +++ b/drivers/net/ethernet/pensando/ionic/ionic_lif.c @@ -299,6 +299,18 @@ static void ionic_lif_qcq_deinit(struct ionic_lif *lif, struct ionic_qcq *qcq) qcq->flags &= ~IONIC_QCQ_F_INITED; } +static void ionic_qcq_intr_free(struct ionic_lif *lif, struct ionic_qcq *qcq) +{ + if (!(qcq->flags & IONIC_QCQ_F_INTR) || qcq->intr.vector == 0) + return; + + irq_set_affinity_hint(qcq->intr.vector, NULL); + devm_free_irq(lif->ionic->dev, qcq->intr.vector, &qcq->napi); + qcq->intr.vector = 0; + ionic_intr_free(lif->ionic, qcq->intr.index); + qcq->intr.index = IONIC_INTR_INDEX_NOT_ASSIGNED; +} + static void ionic_qcq_free(struct ionic_lif *lif, struct ionic_qcq *qcq) { struct device *dev = lif->ionic->dev; @@ -326,12 +338,7 @@ static void ionic_qcq_free(struct ionic_lif *lif, struct ionic_qcq *qcq) qcq->sg_base_pa = 0; } - if (qcq->flags & IONIC_QCQ_F_INTR) { - irq_set_affinity_hint(qcq->intr.vector, NULL); - devm_free_irq(dev, qcq->intr.vector, &qcq->napi); - qcq->intr.vector = 0; - ionic_intr_free(lif->ionic, qcq->intr.index); - } + ionic_qcq_intr_free(lif, qcq); if (qcq->cq.info) { devm_kfree(dev, qcq->cq.info); @@ -341,7 +348,6 @@ static void ionic_qcq_free(struct ionic_lif *lif, struct ionic_qcq *qcq) devm_kfree(dev, qcq->q.info); qcq->q.info = NULL; } - devm_kfree(dev, qcq); } static void ionic_qcqs_free(struct ionic_lif *lif) @@ -350,11 +356,13 @@ static void ionic_qcqs_free(struct ionic_lif *lif) if (lif->notifyqcq) { ionic_qcq_free(lif, lif->notifyqcq); + devm_kfree(dev, lif->notifyqcq); lif->notifyqcq = NULL; } if (lif->adminqcq) { ionic_qcq_free(lif, lif->adminqcq); + devm_kfree(dev, lif->adminqcq); lif->adminqcq = NULL; } @@ -385,6 +393,53 @@ static void ionic_link_qcq_interrupts(struct ionic_qcq *src_qcq, n_qcq->intr.index = src_qcq->intr.index; } +static int ionic_alloc_qcq_interrupt(struct ionic_lif *lif, struct ionic_qcq *qcq) +{ + int err; + + if (!(qcq->flags & IONIC_QCQ_F_INTR)) { + qcq->intr.index = IONIC_INTR_INDEX_NOT_ASSIGNED; + return 0; + } + + err = ionic_intr_alloc(lif, &qcq->intr); + if (err) { + netdev_warn(lif->netdev, "no intr for %s: %d\n", + qcq->q.name, err); + goto err_out; + } + + err = ionic_bus_get_irq(lif->ionic, qcq->intr.index); + if (err < 0) { + netdev_warn(lif->netdev, "no vector for %s: %d\n", + qcq->q.name, err); + goto err_out_free_intr; + } + qcq->intr.vector = err; + ionic_intr_mask_assert(lif->ionic->idev.intr_ctrl, qcq->intr.index, + IONIC_INTR_MASK_SET); + + err = ionic_request_irq(lif, qcq); + if (err) { + netdev_warn(lif->netdev, "irq request failed %d\n", err); + goto err_out_free_intr; + } + + /* try to get the irq on the local numa node first */ + qcq->intr.cpu = cpumask_local_spread(qcq->intr.index, + dev_to_node(lif->ionic->dev)); + if (qcq->intr.cpu != -1) + cpumask_set_cpu(qcq->intr.cpu, &qcq->intr.affinity_mask); + + netdev_dbg(lif->netdev, "%s: Interrupt index %d\n", qcq->q.name, qcq->intr.index); + return 0; + +err_out_free_intr: + ionic_intr_free(lif->ionic, qcq->intr.index); +err_out: + return err; +} + static int ionic_qcq_alloc(struct ionic_lif *lif, unsigned int type, unsigned int index, const char *name, unsigned int flags, @@ -430,39 +485,9 @@ static int ionic_qcq_alloc(struct ionic_lif *lif, unsigned int type, goto err_out_free_q_info; } - if (flags & IONIC_QCQ_F_INTR) { - err = ionic_intr_alloc(lif, &new->intr); - if (err) { - netdev_warn(lif->netdev, "no intr for %s: %d\n", - new->q.name, err); - goto err_out; - } - - err = ionic_bus_get_irq(lif->ionic, new->intr.index); - if (err < 0) { - netdev_warn(lif->netdev, "no vector for %s: %d\n", - new->q.name, err); - goto err_out_free_intr; - } - new->intr.vector = err; - ionic_intr_mask_assert(idev->intr_ctrl, new->intr.index, - IONIC_INTR_MASK_SET); - - err = ionic_request_irq(lif, new); - if (err) { - netdev_warn(lif->netdev, "irq request failed for %s: %d\n", - new->q.name, err); - goto err_out_free_intr; - } - - new->intr.cpu = cpumask_local_spread(new->intr.index, - dev_to_node(dev)); - if (new->intr.cpu != -1) - cpumask_set_cpu(new->intr.cpu, - &new->intr.affinity_mask); - } else { - new->intr.index = IONIC_INTR_INDEX_NOT_ASSIGNED; - } + err = ionic_alloc_qcq_interrupt(lif, new); + if (err) + goto err_out; new->cq.info = devm_kcalloc(dev, num_descs, sizeof(*new->cq.info), GFP_KERNEL); @@ -528,11 +553,10 @@ static int ionic_qcq_alloc(struct ionic_lif *lif, unsigned int type, err_out_free_cq_info: devm_kfree(dev, new->cq.info); err_out_free_irq: - if (flags & IONIC_QCQ_F_INTR) + if (flags & IONIC_QCQ_F_INTR) { devm_free_irq(dev, new->intr.vector, &new->napi); -err_out_free_intr: - if (flags & IONIC_QCQ_F_INTR) ionic_intr_free(lif->ionic, new->intr.index); + } err_out_free_q_info: devm_kfree(dev, new->q.info); err_out_free_qcq: @@ -635,7 +659,7 @@ static int ionic_lif_txq_init(struct ionic_lif *lif, struct ionic_qcq *qcq) unsigned int intr_index; int err; - if (test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state)) + if (qcq->flags & IONIC_QCQ_F_INTR) intr_index = qcq->intr.index; else intr_index = lif->rxqcqs[q->index]->intr.index; @@ -1529,7 +1553,7 @@ static void ionic_txrx_deinit(struct ionic_lif *lif) unsigned int i; if (lif->txqcqs) { - for (i = 0; i < lif->nxqs; i++) { + for (i = 0; i < lif->nxqs && lif->txqcqs[i]; i++) { ionic_lif_qcq_deinit(lif, lif->txqcqs[i]); ionic_tx_flush(&lif->txqcqs[i]->cq); ionic_tx_empty(&lif->txqcqs[i]->q); @@ -1537,7 +1561,7 @@ static void ionic_txrx_deinit(struct ionic_lif *lif) } if (lif->rxqcqs) { - for (i = 0; i < lif->nxqs; i++) { + for (i = 0; i < lif->nxqs && lif->rxqcqs[i]; i++) { ionic_lif_qcq_deinit(lif, lif->rxqcqs[i]); ionic_rx_flush(&lif->rxqcqs[i]->cq); ionic_rx_empty(&lif->rxqcqs[i]->q); @@ -1551,15 +1575,17 @@ static void ionic_txrx_free(struct ionic_lif *lif) unsigned int i; if (lif->txqcqs) { - for (i = 0; i < lif->nxqs; i++) { + for (i = 0; i < lif->ionic->ntxqs_per_lif && lif->txqcqs[i]; i++) { ionic_qcq_free(lif, lif->txqcqs[i]); + devm_kfree(lif->ionic->dev, lif->txqcqs[i]); lif->txqcqs[i] = NULL; } } if (lif->rxqcqs) { - for (i = 0; i < lif->nxqs; i++) { + for (i = 0; i < lif->ionic->nrxqs_per_lif && lif->rxqcqs[i]; i++) { ionic_qcq_free(lif, lif->rxqcqs[i]); + devm_kfree(lif->ionic->dev, lif->rxqcqs[i]); lif->rxqcqs[i] = NULL; } } @@ -2078,20 +2104,22 @@ int ionic_reconfigure_queues(struct ionic_lif *lif, unsigned int i; /* allocate temporary qcq arrays to hold new queue structs */ - if (qparam->ntxq_descs != lif->ntxq_descs) { - tx_qcqs = devm_kcalloc(lif->ionic->dev, lif->nxqs, + if (qparam->nxqs != lif->nxqs || qparam->ntxq_descs != lif->ntxq_descs) { + tx_qcqs = devm_kcalloc(lif->ionic->dev, lif->ionic->ntxqs_per_lif, sizeof(struct ionic_qcq *), GFP_KERNEL); if (!tx_qcqs) goto err_out; } - if (qparam->nrxq_descs != lif->nrxq_descs) { - rx_qcqs = devm_kcalloc(lif->ionic->dev, lif->nxqs, + if (qparam->nxqs != lif->nxqs || qparam->nrxq_descs != lif->nrxq_descs) { + rx_qcqs = devm_kcalloc(lif->ionic->dev, lif->ionic->nrxqs_per_lif, sizeof(struct ionic_qcq *), GFP_KERNEL); if (!rx_qcqs) goto err_out; } - /* allocate new desc_info and rings with no interrupt flag */ + /* allocate new desc_info and rings, but leave the interrupt setup + * until later so as to not mess with the still-running queues + */ if (lif->qtype_info[IONIC_QTYPE_TXQ].version >= 1 && lif->qtype_info[IONIC_QTYPE_TXQ].sg_desc_sz == sizeof(struct ionic_txq_sg_desc_v1)) @@ -2100,7 +2128,7 @@ int ionic_reconfigure_queues(struct ionic_lif *lif, sg_desc_sz = sizeof(struct ionic_txq_sg_desc); if (tx_qcqs) { - for (i = 0; i < lif->nxqs; i++) { + for (i = 0; i < qparam->nxqs; i++) { flags = lif->txqcqs[i]->flags & ~IONIC_QCQ_F_INTR; err = ionic_qcq_alloc(lif, IONIC_QTYPE_TXQ, i, "tx", flags, qparam->ntxq_descs, @@ -2114,7 +2142,7 @@ int ionic_reconfigure_queues(struct ionic_lif *lif, } if (rx_qcqs) { - for (i = 0; i < lif->nxqs; i++) { + for (i = 0; i < qparam->nxqs; i++) { flags = lif->rxqcqs[i]->flags & ~IONIC_QCQ_F_INTR; err = ionic_qcq_alloc(lif, IONIC_QTYPE_RXQ, i, "rx", flags, qparam->nrxq_descs, @@ -2136,21 +2164,70 @@ int ionic_reconfigure_queues(struct ionic_lif *lif, /* swap new desc_info and rings, keeping existing interrupt config */ if (tx_qcqs) { lif->ntxq_descs = qparam->ntxq_descs; - for (i = 0; i < lif->nxqs; i++) + for (i = 0; i < qparam->nxqs; i++) ionic_swap_queues(lif->txqcqs[i], tx_qcqs[i]); } if (rx_qcqs) { lif->nrxq_descs = qparam->nrxq_descs; - for (i = 0; i < lif->nxqs; i++) + for (i = 0; i < qparam->nxqs; i++) ionic_swap_queues(lif->rxqcqs[i], rx_qcqs[i]); } + /* if we need to change the interrupt layout, this is the time */ + if (qparam->intr_split != test_bit(IONIC_LIF_F_SPLIT_INTR, lif->state) || + qparam->nxqs != lif->nxqs) { + if (qparam->intr_split) { + set_bit(IONIC_LIF_F_SPLIT_INTR, lif->state); + } else { + clear_bit(IONIC_LIF_F_SPLIT_INTR, lif->state); + lif->tx_coalesce_usecs = lif->rx_coalesce_usecs; + lif->tx_coalesce_hw = lif->rx_coalesce_hw; + } + + /* clear existing interrupt assignments */ + for (i = 0; i < lif->ionic->ntxqs_per_lif; i++) { + ionic_qcq_intr_free(lif, lif->txqcqs[i]); + ionic_qcq_intr_free(lif, lif->rxqcqs[i]); + } + + /* re-assign the interrupts */ + for (i = 0; i < qparam->nxqs; i++) { + lif->rxqcqs[i]->flags |= IONIC_QCQ_F_INTR; + err = ionic_alloc_qcq_interrupt(lif, lif->rxqcqs[i]); + ionic_intr_coal_init(lif->ionic->idev.intr_ctrl, + lif->rxqcqs[i]->intr.index, + lif->rx_coalesce_hw); + + if (qparam->intr_split) { + lif->txqcqs[i]->flags |= IONIC_QCQ_F_INTR; + err = ionic_alloc_qcq_interrupt(lif, lif->txqcqs[i]); + ionic_intr_coal_init(lif->ionic->idev.intr_ctrl, + lif->txqcqs[i]->intr.index, + lif->tx_coalesce_hw); + } else { + lif->txqcqs[i]->flags &= ~IONIC_QCQ_F_INTR; + ionic_link_qcq_interrupts(lif->rxqcqs[i], lif->txqcqs[i]); + } + } + } + + swap(lif->nxqs, qparam->nxqs); + /* re-init the queues */ err = ionic_txrx_init(lif); if (err) goto err_out_attach_unlock; + if (qparam->nxqs != lif->nxqs) { + err = netif_set_real_num_tx_queues(lif->netdev, lif->nxqs); + if (err) + goto err_out; + err = netif_set_real_num_rx_queues(lif->netdev, lif->nxqs); + if (err) + goto err_out; + } + /* don't start the queues unless we have link */ if (netif_carrier_ok(lif->netdev)) { err = ionic_start_queues(lif); @@ -2164,15 +2241,17 @@ int ionic_reconfigure_queues(struct ionic_lif *lif, err_out: /* free old allocs without cleaning intr */ - for (i = 0; i < lif->nxqs; i++) { + for (i = 0; i < qparam->nxqs; i++) { if (tx_qcqs && tx_qcqs[i]) { tx_qcqs[i]->flags &= ~IONIC_QCQ_F_INTR; ionic_qcq_free(lif, tx_qcqs[i]); + devm_kfree(lif->ionic->dev, tx_qcqs[i]); tx_qcqs[i] = NULL; } if (rx_qcqs && rx_qcqs[i]) { rx_qcqs[i]->flags &= ~IONIC_QCQ_F_INTR; ionic_qcq_free(lif, rx_qcqs[i]); + devm_kfree(lif->ionic->dev, rx_qcqs[i]); rx_qcqs[i] = NULL; } } @@ -2187,6 +2266,17 @@ int ionic_reconfigure_queues(struct ionic_lif *lif, tx_qcqs = NULL; } + /* clean the unused dma and info allocations when new set is smaller + * than the full array, but leave the qcq shells in place + */ + for (i = lif->nxqs; i < lif->ionic->ntxqs_per_lif; i++) { + lif->txqcqs[i]->flags &= ~IONIC_QCQ_F_INTR; + ionic_qcq_free(lif, lif->txqcqs[i]); + + lif->rxqcqs[i]->flags &= ~IONIC_QCQ_F_INTR; + ionic_qcq_free(lif, lif->rxqcqs[i]); + } + return err; }