From patchwork Fri Sep 10 23:31:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 509355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F90CC433EF for ; Fri, 10 Sep 2021 23:32:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 11D5E61206 for ; Fri, 10 Sep 2021 23:32:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234806AbhIJXdV (ORCPT ); Fri, 10 Sep 2021 19:33:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234789AbhIJXdU (ORCPT ); Fri, 10 Sep 2021 19:33:20 -0400 Received: from mail-pl1-x630.google.com (mail-pl1-x630.google.com [IPv6:2607:f8b0:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02A97C061574 for ; Fri, 10 Sep 2021 16:32:09 -0700 (PDT) Received: by mail-pl1-x630.google.com with SMTP id c4so582987pls.6 for ; Fri, 10 Sep 2021 16:32:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vCHNRfaj2dKGgARCTQjE9Cv8ooW2hX6jrMwBXg6VlYQ=; b=expjbiICBtnT3J4yUyWnm1J4/2KQBuiLlz2j2I8lkhelCF/RDE1G04KJh8iEZx8eqz TY+79QYtLf94wg5djLI/fMq8rdyUcZJnHhYHNToChu05dTDP5UCUXV8lLlkj+ReJEhTY IfRBENT5yohYY4F5e/AZetQ4qxoM1XONGGtcLkFQK7E1a4fySDZMVWbbPGCFMLcV6fKP 418bvYjKr8oNXVulMFzw+nU5CKKCFCRgbh+w4DLbJ83fJOQ8JeZ7Sh+2e55l8KKUuCF3 igGOQzV+Auv4JpOK63rN948BUC3zFcyTu/pyNLMYE0dVTqRTGkrAUgxvvI22vj+QS/oq 9v6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vCHNRfaj2dKGgARCTQjE9Cv8ooW2hX6jrMwBXg6VlYQ=; b=x7IjoHIzxXhDVnf3WtqWZnhhgsO1FzrdVehFI0M8uRZrZs9NCMR5gAWHSWuYVPI4jl cy5ZN7dAdFQorGVny2g9hr+v0EQ6GaKNuvcUOAwT1iGO2AjkZ5wF31TL4aOCOYu7uIa0 vIIv6l1818YEs5yPe6PnSFslU7phYOQhikPVFv5e8MWEr+/UoFE5AV7nOIBiHqFwRPH/ dBRPz4n1+z9JqEKArPPwzpkuQgzHTnzUCHC+m4yjfOUpykue8V5Xl8EnkqzvzVTvnszy iQvF2klTSo2FcRFHSJ8W6l6Imkw4lJhvwVFjF9nB1OsglscfT/u+uMJZ5P40wimY4P8w PPMw== X-Gm-Message-State: AOAM530VYFDBbHbbwNqX5XkoaFEoHeztQQk9wYjz1NTXLYwpFHaQPs0N EE72Sg+7VxH2fQ0gkc0b+MXr9kEvF9VaLX92 X-Google-Smtp-Source: ABdhPJwnxbBkf+7Kszh+aJmhIO5T9KZVwGsVtxjhWz813/CzCgv+MM/RrvP455xYaVv4PShJ6YAE2A== X-Received: by 2002:a17:90b:33c8:: with SMTP id lk8mr112020pjb.241.1631316728434; Fri, 10 Sep 2021 16:32:08 -0700 (PDT) Received: from mail-ash-it-01.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id o15sm11325pfk.143.2021.09.10.16.32.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Sep 2021 16:32:07 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH 01/14] lpfc: Fix list_add corruption in lpfc_drain_txq Date: Fri, 10 Sep 2021 16:31:46 -0700 Message-Id: <20210910233159.115896-2-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210910233159.115896-1-jsmart2021@gmail.com> References: <20210910233159.115896-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org When parsing the txq list in lpfc_drain_txq, the driver attempts to pass the requests to the adapter. If such an attempt fails, a local "fail_msg" string is set and a log message output. The job is then added to a completions list for cancellation. Processing of any further jobs from the txq list continues, but since "fail_msg" remains set, jobs are added to the completions list regardless of whether a wqe was passed to the adapter. If successfully added to txcmplq, jobs are added to both lists resulting in list corruption. Fix by clearing the fail_msg string after adding a job to the completions list. This stops the subsequent jobs from being added to the completions list unless they had an appropriate failure. Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_sli.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index ffd8a140638c..546c851938bc 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -21104,6 +21104,7 @@ lpfc_drain_txq(struct lpfc_hba *phba) fail_msg, piocbq->iotag, piocbq->sli4_xritag); list_add_tail(&piocbq->list, &completions); + fail_msg = NULL; } spin_unlock_irqrestore(&pring->ring_lock, iflags); } From patchwork Fri Sep 10 23:31:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 509006 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C7A3EC433F5 for ; Fri, 10 Sep 2021 23:32:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A845861213 for ; Fri, 10 Sep 2021 23:32:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234860AbhIJXdV (ORCPT ); Fri, 10 Sep 2021 19:33:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38528 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234854AbhIJXdV (ORCPT ); Fri, 10 Sep 2021 19:33:21 -0400 Received: from mail-pj1-x102f.google.com (mail-pj1-x102f.google.com [IPv6:2607:f8b0:4864:20::102f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A26D5C061574; Fri, 10 Sep 2021 16:32:09 -0700 (PDT) Received: by mail-pj1-x102f.google.com with SMTP id dw14so1492496pjb.1; Fri, 10 Sep 2021 16:32:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uNt0QAxuKO6+FYL7vNETK+bdI7sWgzN3S59lbqChJtc=; b=lcx2UnmquHfH3ywNLVF34isiqgeMqqpjdbgYEak5/bS/b0YcTT5JLK3ZSeg+vK89lK IpOXdGjD1CtfviBroZ2MSX7OhWyyg2nUUNbr0amghI2gN+D9MlqqoLTsTl4dQRSuUhD0 8IQx6eQfgq+CuSmjV5GyP8qwPwTlaXrvNvT9FmjnCXoLrkh5rpZ8m6YeMt0LAX42M+bq JwAoUjvcsuvz8xdPgSouLjlLfjZPG2he7Kz3jsGsx2zf2uS1jD2yvWVrz1if19tNFREF mCKBit7q8CWrcB/nNh8IzZw8xS/JLPa08RyAQDitnBim9Y5Cc84aUFGa/6SWlfZLA8pk fVtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uNt0QAxuKO6+FYL7vNETK+bdI7sWgzN3S59lbqChJtc=; b=3U7CeFJ7sBavGS0w+sR0/r76jAFMwXa5OGCw4As4q9IeKtBcjtGX73/eI7+5hU2ffh exxdXqucis5Rmmb+3BanJXAMZeAxh+ZyGxdC40eiT70AkYB0RO1PpjWbJetW48bhx2Yn +vORR4DbfC4D9eomRN0SSIwqRfOXYW/oofx5NFk8CsdgqMFUBPw9/LQN+CfYioJwzHdQ Va/eYgbJeUTgI9pqaF00HNSoU4XbExNFb3LwasgI7gq0n/tTJ37rtt/ytPR7P/qIGEr8 XoeIrLLCN9o5HUbsl1oSxTOrgiBQFVKF0iXEWSo9rVoqM3D+g6snEwcv4fMJargHDOc5 ZC1A== X-Gm-Message-State: AOAM530Vie2uR4J7Ut2BJZtnRnVP940pvP4FBg/7rng+1V0h4pUWiJo7 BHz463289Z/tPHCXIMatHXlx1QNWavaStV4r X-Google-Smtp-Source: ABdhPJyKVTQpRmHNQ6eD/rUxxDXSgW3HSQzloNtA0zUSUuO+6VE7pcYlbIRSawMmCzVTnwyQnnrV7A== X-Received: by 2002:a17:90b:3a8e:: with SMTP id om14mr147644pjb.192.1631316729039; Fri, 10 Sep 2021 16:32:09 -0700 (PDT) Received: from mail-ash-it-01.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id o15sm11325pfk.143.2021.09.10.16.32.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Sep 2021 16:32:08 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , stable@vger.kernel.org, Justin Tee Subject: [PATCH 02/14] lpfc: Don't release final kref on Fport node while ABTS outstanding Date: Fri, 10 Sep 2021 16:31:47 -0700 Message-Id: <20210910233159.115896-3-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210910233159.115896-1-jsmart2021@gmail.com> References: <20210910233159.115896-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In a rarely executed path, FLOGI failure, there is a refcounting error. If FLOGI completed with an error, typically a timeout, the initial completion handler would remove the job reference. However, the job completion isn't the actual end of the job/exchange as the timeout usually initiates an ABTS, and upon that ABTS completion, a final completion is sent. The driver removes the reference again in the final completion. Thus the imbalance. In the buggy cases, if there was a link bounce while the delayed response is outstanding, the fport node may be referenced again but there was no additional reference as it is already present. The delayed completion then occurs and removes the last reference freeing the node and causing issues in the link up processed that is using the node. Fix this scenario by removing the snippet that removed the reference in the initial flogi completion. The bad snippet was poorly trying to identify the flogi as ok to do so by realizing the node was not registered with either SCSI or NVME transport. Fixes: 618e2ee146d4 ("scsi: lpfc: Fix FLOGI failure due to accessing a freed node") Cc: # v5.13+ Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_els.c | 11 +++++------ drivers/scsi/lpfc/lpfc_hbadisc.c | 10 ++++++---- drivers/scsi/lpfc/lpfc_nvme.c | 5 +++-- 3 files changed, 14 insertions(+), 12 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c index 1254a575fd47..df5fc223ddb2 100644 --- a/drivers/scsi/lpfc/lpfc_els.c +++ b/drivers/scsi/lpfc/lpfc_els.c @@ -1059,9 +1059,10 @@ lpfc_cmpl_els_flogi(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, lpfc_printf_vlog(vport, KERN_WARNING, LOG_TRACE_EVENT, "0150 FLOGI failure Status:x%x/x%x " - "xri x%x TMO:x%x\n", + "xri x%x TMO:x%x refcnt %d\n", irsp->ulpStatus, irsp->un.ulpWord[4], - cmdiocb->sli4_xritag, irsp->ulpTimeout); + cmdiocb->sli4_xritag, irsp->ulpTimeout, + kref_read(&ndlp->kref)); /* If this is not a loop open failure, bail out */ if (!(irsp->ulpStatus == IOSTAT_LOCAL_REJECT && @@ -1122,12 +1123,12 @@ lpfc_cmpl_els_flogi(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, /* FLOGI completes successfully */ lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS, "0101 FLOGI completes successfully, I/O tag:x%x, " - "xri x%x Data: x%x x%x x%x x%x x%x x%x x%x\n", + "xri x%x Data: x%x x%x x%x x%x x%x x%x x%x %d\n", cmdiocb->iotag, cmdiocb->sli4_xritag, irsp->un.ulpWord[4], sp->cmn.e_d_tov, sp->cmn.w2.r_a_tov, sp->cmn.edtovResolution, vport->port_state, vport->fc_flag, - sp->cmn.priority_tagging); + sp->cmn.priority_tagging, kref_read(&ndlp->kref)); if (sp->cmn.priority_tagging) vport->vmid_flag |= LPFC_VMID_ISSUE_QFPA; @@ -1205,8 +1206,6 @@ lpfc_cmpl_els_flogi(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, phba->fcf.fcf_flag &= ~FCF_DISCOVERY; spin_unlock_irq(&phba->hbalock); - if (!(ndlp->fc4_xpt_flags & (SCSI_XPT_REGD | NVME_XPT_REGD))) - lpfc_nlp_put(ndlp); if (!lpfc_error_lost_link(irsp)) { /* FLOGI failed, so just use loop map to make discovery list */ lpfc_disc_list_loopmap(vport); diff --git a/drivers/scsi/lpfc/lpfc_hbadisc.c b/drivers/scsi/lpfc/lpfc_hbadisc.c index 7195ca0275f9..6f2e07c30f98 100644 --- a/drivers/scsi/lpfc/lpfc_hbadisc.c +++ b/drivers/scsi/lpfc/lpfc_hbadisc.c @@ -4449,8 +4449,9 @@ lpfc_register_remote_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) fc_remote_port_rolechg(rport, rport_ids.roles); lpfc_printf_vlog(ndlp->vport, KERN_INFO, LOG_NODE, - "3183 %s rport x%px DID x%x, role x%x\n", - __func__, rport, rport->port_id, rport->roles); + "3183 %s rport x%px DID x%x, role x%x refcnt %d\n", + __func__, rport, rport->port_id, rport->roles, + kref_read(&ndlp->kref)); if ((rport->scsi_target_id != -1) && (rport->scsi_target_id < LPFC_MAX_TARGET)) { @@ -4475,8 +4476,9 @@ lpfc_unregister_remote_port(struct lpfc_nodelist *ndlp) lpfc_printf_vlog(vport, KERN_INFO, LOG_NODE, "3184 rport unregister x%06x, rport x%px " - "xptflg x%x\n", - ndlp->nlp_DID, rport, ndlp->fc4_xpt_flags); + "xptflg x%x refcnt %d\n", + ndlp->nlp_DID, rport, ndlp->fc4_xpt_flags, + kref_read(&ndlp->kref)); fc_remote_port_delete(rport); lpfc_nlp_put(ndlp); diff --git a/drivers/scsi/lpfc/lpfc_nvme.c b/drivers/scsi/lpfc/lpfc_nvme.c index 73a3568ff17e..bd88477f9b82 100644 --- a/drivers/scsi/lpfc/lpfc_nvme.c +++ b/drivers/scsi/lpfc/lpfc_nvme.c @@ -209,8 +209,9 @@ lpfc_nvme_remoteport_delete(struct nvme_fc_remote_port *remoteport) * calling state machine to remove the node. */ lpfc_printf_vlog(vport, KERN_INFO, LOG_NVME_DISC, - "6146 remoteport delete of remoteport x%px\n", - remoteport); + "6146 remoteport delete of remoteport x%px, ndlp x%px " + "DID x%x xflags x%x\n", + remoteport, ndlp, ndlp->nlp_DID, ndlp->fc4_xpt_flags); spin_lock_irq(&ndlp->lock); /* The register rebind might have occurred before the delete From patchwork Fri Sep 10 23:31:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 509354 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01931C433F5 for ; Fri, 10 Sep 2021 23:32:17 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D9B2C61206 for ; Fri, 10 Sep 2021 23:32:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234874AbhIJXdY (ORCPT ); Fri, 10 Sep 2021 19:33:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38536 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234859AbhIJXdV (ORCPT ); Fri, 10 Sep 2021 19:33:21 -0400 Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 60C4DC061574 for ; Fri, 10 Sep 2021 16:32:10 -0700 (PDT) Received: by mail-pl1-x62f.google.com with SMTP id bg1so2110067plb.13 for ; Fri, 10 Sep 2021 16:32:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uEVsuIX/m/sL25A3lEyvCf2DS8cjQrVPW94B/QPsGw8=; b=TcXSeEizhUwBrV/v/MyqUWkN3uLoH2O6ksyxa8wc+3ZL0sFWY7YAglBsrVwv9Wevpo U+P1/RrrzhagYVk1ageIPoYqRzwIesD5mORuO4+XRoxHviMkQPbfeWfDDXx1IJ7n2E1o rVp5TsVtUE0tTz0UsfIbUwXJRh2u0UOGKZci3+VKJsBHORkbUW79l4reyWsNM5RA+ISY 1D4ocqV++XtGpawCLyqLx3qJpHLfADf7uDKVsINwCFigyhxoGFAFcQuWOx4xtHZQtFZE cjGQ+YqULwyro06vrCY/WcjpCzbkjdgbsrwmRDlvNhNZ4Mkwjcdfs0Aigm6itc4pIxQ9 8WIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uEVsuIX/m/sL25A3lEyvCf2DS8cjQrVPW94B/QPsGw8=; b=a/uS511lIndsH9kfIdhX+tazguvzqQf3pWbdVDTfWYbWVaBhNalxieQC/RAsdXlYxg 0KddjRqzDSF3FKyvHH6ugFqwKsbE09yprt/qlXowo7uG11/FsJZf6Uy2xVmtgT2hf3ea c/NikAaJ+1Jk8wjvU+5dFdtkV5k/4mWU+dRUqpa03l1/ZQUpCJRPzXJq0eY9q+KD4utO vGBvUPLzmWPaZ3Hl3xHKeCc27Bu6MejidoFcx4lr9jpK6bS8cTD23QrjAARXohOccgQj fBHwdRn4UTQS5F8FQHMuRKYCBHHlfMhGyT+8wpiAK+/21bXQv+p9CuU6Z7o62b0M5K9n UnDQ== X-Gm-Message-State: AOAM531JLhUwYtQ+FjwfVtjt+BNHSplHev841eHRVvChxIPlwBHoOErE 4ZtqMpaG17ahIdFFvbpD4XT8NpUQlFUYIqtM X-Google-Smtp-Source: ABdhPJzwo70Ts39NKgnRy9thg+0Y1hj3jtQz3s+7tBqgOT2l2PIboqXhd6r8XdHQKrNTHtTc6gHzFw== X-Received: by 2002:a17:90a:990b:: with SMTP id b11mr83375pjp.182.1631316729700; Fri, 10 Sep 2021 16:32:09 -0700 (PDT) Received: from mail-ash-it-01.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id o15sm11325pfk.143.2021.09.10.16.32.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Sep 2021 16:32:09 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH 03/14] lpfc: Fix premature rpi release for unsolicited TPLS and LS_RJT Date: Fri, 10 Sep 2021 16:31:48 -0700 Message-Id: <20210910233159.115896-4-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210910233159.115896-1-jsmart2021@gmail.com> References: <20210910233159.115896-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org A test scenario has a target issuing a TPLS after accepting the driver's PRLI. TPLS is not supported by the driver so it rejects the ELS. However, the reject was only happening on the primary N_Port. If the TPLS was to a NPIV vport, not only would it reject the ELS, but it would act on the TPLS, starting devloss, then unregister from the SCSI transport and release the node. When devloss expired, it would access the node again and cause a page faul. Fix by altering the NPIV code to recognize that a correctly registered node can reject unsolicited ELS IO and to not unregister with the SCSI transport and tear the node down. Add a check of the fc4_xpt_flags so that only a zero value allows the unreg and teardown. Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_els.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c index df5fc223ddb2..262101e172ad 100644 --- a/drivers/scsi/lpfc/lpfc_els.c +++ b/drivers/scsi/lpfc/lpfc_els.c @@ -5295,6 +5295,7 @@ lpfc_cmpl_els_rsp(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, */ if (phba->sli_rev == LPFC_SLI_REV4 && (vport && vport->port_type == LPFC_NPIV_PORT) && + !(ndlp->fc4_xpt_flags & SCSI_XPT_REGD) && ndlp->nlp_flag & NLP_RELEASE_RPI) { lpfc_sli4_free_rpi(phba, ndlp->nlp_rpi); spin_lock_irq(&ndlp->lock); @@ -5598,11 +5599,12 @@ lpfc_els_rsp_reject(struct lpfc_vport *vport, uint32_t rejectError, } /* The NPIV instance is rejecting this unsolicited ELS. Make sure the - * node's assigned RPI needs to be released as this node will get - * freed. + * node's assigned RPI gets released provided this node is not already + * registered with the transport. */ if (phba->sli_rev == LPFC_SLI_REV4 && - vport->port_type == LPFC_NPIV_PORT) { + vport->port_type == LPFC_NPIV_PORT && + !(ndlp->fc4_xpt_flags & SCSI_XPT_REGD)) { spin_lock_irq(&ndlp->lock); ndlp->nlp_flag |= NLP_RELEASE_RPI; spin_unlock_irq(&ndlp->lock); From patchwork Fri Sep 10 23:31:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 509005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3D097C433EF for ; Fri, 10 Sep 2021 23:32:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 24776611B0 for ; Fri, 10 Sep 2021 23:32:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234890AbhIJXd2 (ORCPT ); Fri, 10 Sep 2021 19:33:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38538 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234854AbhIJXdW (ORCPT ); Fri, 10 Sep 2021 19:33:22 -0400 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E4BB9C061756 for ; Fri, 10 Sep 2021 16:32:10 -0700 (PDT) Received: by mail-pf1-x429.google.com with SMTP id j16so3259202pfc.2 for ; Fri, 10 Sep 2021 16:32:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TB6O2EeP1WftlXbEahfA7Ed8vQ5inRUvqVbgSRnWOvI=; b=VfyvuaXOzyshUFbqny33HO/SMj0KJbP5bC66BzQb57U4m4pE0W0Nv0NU+GlCC3aIti HzNJft5hiy3/DhsC+qF5wteKrrgrPgU+lEZOJLeX3mYRdj0a90gY3BZokTZPzli/7QM7 Q2NVcakRy/+5h7QWarI8N4lBZUOkstDg8rMhlKYfYCXqphhVhZMAl7lpawTc+poWhPyU JNj3c3teyB5A4AU9N7ylSYJ1iCNdXDKsnkjzmMdbfKiLXY8QDI9E64NSFklMLsDdXobK Cjt+mLkbEi8EHOo1s7YViC7m/UvXaWnNY4HTBZmLBEf62QRN61dyyaP62clMLveK8n93 mAmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TB6O2EeP1WftlXbEahfA7Ed8vQ5inRUvqVbgSRnWOvI=; b=RHRsgBJjDVxxc9BIlgk09rz4/ON1IX+pa3uxzCZuP3tG8FBGqj6WXCaMkMdC3jkzi+ UopD5x5z5PmXRwk7u3zOO2tDfsHFIRWuSU+FGzsCFuNWkFWdaxzuDfeKQ4JI3K8IrL0d BzkeBkWRBfr9DfvZgUGh5+AJOhUYweOd3kC4iY32DYuQpz88csnArb3kJUMmlBv8hMMC rsYC4W0cAxZU1IhFIvPxkpuaUw/0ZXFuKELnWDjXi7uh0aHygNMAU3STQcbGMvLt1oAA 6L3YKN5isO5NbizHQfta/XtIuaaQm9kDYyWNNG1n4edSp0d4Jv+yxDcuP4VBlm35Yi2L vpkw== X-Gm-Message-State: AOAM530SClK4d8hgdSFzB6F5VSGHXZKTJKea8G5maeAJi3E7yMCk/a37 DWVKuie/ClI0wgyHKG7RXFKlRlh7UrA2IWCx X-Google-Smtp-Source: ABdhPJwjzQj+bYG2hDCSIyiqdH9DtYP/b+XYm0thRf++XXG3S0hN3Oa86yEC/xCEmYyBKc3BryoK4w== X-Received: by 2002:a63:da0a:: with SMTP id c10mr151954pgh.255.1631316730380; Fri, 10 Sep 2021 16:32:10 -0700 (PDT) Received: from mail-ash-it-01.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id o15sm11325pfk.143.2021.09.10.16.32.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Sep 2021 16:32:10 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH 04/14] lpfc: Fix hang on unload due to stuck fport node Date: Fri, 10 Sep 2021 16:31:49 -0700 Message-Id: <20210910233159.115896-5-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210910233159.115896-1-jsmart2021@gmail.com> References: <20210910233159.115896-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org A test scenario encountered an unload hang while an FLOGI ELS was in flight when a link down condition occurred. The driver fails unload as it never releases the fport node. For most nodes, when the link drops, devloss tmo is started and the timeout will cause the final node release. For the Fport, as it has not yet registered with the SCSI transport, there is no devloss timer to be started, so there is no final release. Additionally, the link down sequence causes ABORTS to be issued for pending ELS's. The completions from the ABORTS perform the release of node references. However, as the adapter is being reset to be unloaded, those completions will never occur. Fix by the following: - In the els cleanup, recognize when unloading and place the els's on a different list that immediately cleansup/completes the els's. It's recognized that this condition primarily affects only the fport, with other ports having normal clean up logic that handles things. - Resolve the devloss issue by, when cleaning up nodes on after link down, recognizing when the fabric node does not have a completed state (its state is UNUSED) and removing a reference so the node can delete after the els reference is released. Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_els.c | 14 ++++++++++++++ drivers/scsi/lpfc/lpfc_hbadisc.c | 14 +++++++++++++- 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c index 262101e172ad..6c9cb87ef174 100644 --- a/drivers/scsi/lpfc/lpfc_els.c +++ b/drivers/scsi/lpfc/lpfc_els.c @@ -11386,6 +11386,7 @@ lpfc_sli4_vport_delete_els_xri_aborted(struct lpfc_vport *vport) { struct lpfc_hba *phba = vport->phba; struct lpfc_sglq *sglq_entry = NULL, *sglq_next = NULL; + struct lpfc_nodelist *ndlp = NULL; unsigned long iflag = 0; spin_lock_irqsave(&phba->sli4_hba.sgl_list_lock, iflag); @@ -11393,7 +11394,20 @@ lpfc_sli4_vport_delete_els_xri_aborted(struct lpfc_vport *vport) &phba->sli4_hba.lpfc_abts_els_sgl_list, list) { if (sglq_entry->ndlp && sglq_entry->ndlp->vport == vport) { lpfc_nlp_put(sglq_entry->ndlp); + ndlp = sglq_entry->ndlp; sglq_entry->ndlp = NULL; + + /* If the xri on the abts_els_sgl list is for the Fport + * node and the vport is unloading, the xri aborted wcqe + * likely isn't coming back. Just release the sgl. + */ + if ((vport->load_flag & FC_UNLOADING) && + ndlp->nlp_DID == Fabric_DID) { + list_del(&sglq_entry->list); + sglq_entry->state = SGL_FREED; + list_add_tail(&sglq_entry->list, + &phba->sli4_hba.lpfc_els_sgl_list); + } } } spin_unlock_irqrestore(&phba->sli4_hba.sgl_list_lock, iflag); diff --git a/drivers/scsi/lpfc/lpfc_hbadisc.c b/drivers/scsi/lpfc/lpfc_hbadisc.c index 6f2e07c30f98..4ff93aef3295 100644 --- a/drivers/scsi/lpfc/lpfc_hbadisc.c +++ b/drivers/scsi/lpfc/lpfc_hbadisc.c @@ -966,8 +966,20 @@ lpfc_cleanup_rpis(struct lpfc_vport *vport, int remove) struct lpfc_nodelist *ndlp, *next_ndlp; list_for_each_entry_safe(ndlp, next_ndlp, &vport->fc_nodes, nlp_listp) { - if (ndlp->nlp_state == NLP_STE_UNUSED_NODE) + if (ndlp->nlp_state == NLP_STE_UNUSED_NODE) { + /* It's possible the FLOGI to the fabric node never + * successfully completed and never registered with the + * transport. In this case there is no way to clean up + * the node. + */ + if (ndlp->nlp_DID == Fabric_DID) { + if (ndlp->nlp_prev_state == + NLP_STE_UNUSED_NODE && + !ndlp->fc4_xpt_flags) + lpfc_nlp_put(ndlp); + } continue; + } if ((phba->sli3_options & LPFC_SLI3_VPORT_TEARDOWN) || ((vport->port_type == LPFC_NPIV_PORT) && From patchwork Fri Sep 10 23:31:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 509353 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8CAAC433EF for ; Fri, 10 Sep 2021 23:32:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A3732611B0 for ; Fri, 10 Sep 2021 23:32:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234882AbhIJXdg (ORCPT ); Fri, 10 Sep 2021 19:33:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38570 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234883AbhIJXd0 (ORCPT ); Fri, 10 Sep 2021 19:33:26 -0400 Received: from mail-pj1-x1029.google.com (mail-pj1-x1029.google.com [IPv6:2607:f8b0:4864:20::1029]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA409C06175F for ; Fri, 10 Sep 2021 16:32:14 -0700 (PDT) Received: by mail-pj1-x1029.google.com with SMTP id lb1-20020a17090b4a4100b001993f863df2so1679091pjb.5 for ; Fri, 10 Sep 2021 16:32:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=B4PFJ+5cXtDvpz0WDtpK4XiTZaISaTldxnUnweyif6c=; b=peNpi04xILByRK5hCVEQpwZ7zchpPTyT1dYemI58pTjCVcAQ3HEAyjxf1JBabg2DoL +ZK7zKh6zkKcc1A63QTD27YpIO6J49dxtJgNIDEYgtJHr8Bufr5VHdftIEhs/j0nEG4C YKEh2mCMKOwpqvHcKqi0Mp1zA8kd6tqCglEvZV0ID+Xs8T3ozXsuBIV4P1VZnlAKAE9I tcx45EkdZQEdNZy37ef49L51YeBvQpGqD0M0i1QoCCr7xg8wFWvnEaHPrUAfEXsW843i YwRSXSIYM9XyTW28qXRJrRNsceoaUOnR57jTpIJcQ1lZiUChv39KXktJoGTy3mPqkhId eTNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=B4PFJ+5cXtDvpz0WDtpK4XiTZaISaTldxnUnweyif6c=; b=PCoVGg66tEondDaujvhhJzbpMn3eMKnQXVt4ruGSKANSMKnsm5cil8SLZJJvRs1bkN ctCv3BqEHeZDc/I0WUHuDZroAmuXMxJ35ZMEfbT/jXoBTwI+vzP8UlrJ1gXrh+axkRHU F6MsOVGQxyBkaxevszhNZRyzuH/Hl0BuSk57TSkL6tIOr4SMiJ7lZRG92pMP/BrifsGN 3nRuUFtMxbB0/Clyi32Hx4Hk5MO8WWjAxxhRVo5GK/LwtnFqlvwP/AfPwVqDdpslMnTQ U7Vi8PpDRc6JnkKStsvmAGjHOVow8Ui7FUCHpTj3kNU3fMVheBZwZv8N5FavjRKEo32H YkTQ== X-Gm-Message-State: AOAM533udNpXJo8Hk87xTLp2f8Ai5jHsKUCCL41ANHeAvVDow2MMrpHR R9PNnAMFYxxi4KSBlHJbYhO8GU3og6D9R8gV X-Google-Smtp-Source: ABdhPJzB9YZTV/HIqk5TaH87bN4xL069C/EMnF6JVm1sz5qTD31JQfa6pjylx+I/eQYI2cxzalF32Q== X-Received: by 2002:a17:902:b691:b029:12d:2b6:d116 with SMTP id c17-20020a170902b691b029012d02b6d116mr105564pls.71.1631316734090; Fri, 10 Sep 2021 16:32:14 -0700 (PDT) Received: from mail-ash-it-01.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id o15sm11325pfk.143.2021.09.10.16.32.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Sep 2021 16:32:10 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH 05/14] lpfc: Fix rediscovery of tape device after issue lip Date: Fri, 10 Sep 2021 16:31:50 -0700 Message-Id: <20210910233159.115896-6-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210910233159.115896-1-jsmart2021@gmail.com> References: <20210910233159.115896-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org On link up and node discovery, a remote port is registered with the scsi transport and the driver sets fc4_xpt_flags to track transport registration. A link down event causes the driver to deregister with the scsi transport, starting the devloss timer, and calls a local unreg routine to clear the login state. Part of the login state is the fc4_xpt_flags. However, with TAPE devices that support sequence level error recovery, which wants to preserve the login, the local unreg routine is skipped, thus the flags aren't cleared. A subsequent link up, ADISC is performed and the lpfc_nlp_reg_node() routine is called. As the fc4_xpt_flags is not clear, it's believed the node is already registered with the transport. Unfortunately, the registration was already terminated. Eventually the devloss tmo timer expires and tears down the device. Fix by ensuring the tape device, known by the ADISC flag, is always unregistered if the link drops. Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_hbadisc.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_hbadisc.c b/drivers/scsi/lpfc/lpfc_hbadisc.c index 4ff93aef3295..12abfc027a67 100644 --- a/drivers/scsi/lpfc/lpfc_hbadisc.c +++ b/drivers/scsi/lpfc/lpfc_hbadisc.c @@ -4693,8 +4693,11 @@ lpfc_nlp_state_cleanup(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp, /* Reg/Unreg for FCP and NVME Transport interface */ if ((old_state == NLP_STE_MAPPED_NODE || old_state == NLP_STE_UNMAPPED_NODE)) { - /* For nodes marked for ADISC, Handle unreg in ADISC cmpl */ - if (!(ndlp->nlp_flag & NLP_NPR_ADISC)) + /* For nodes marked for ADISC, Handle unreg in ADISC cmpl + * if linkup. In linkdown do unreg_node + */ + if (!(ndlp->nlp_flag & NLP_NPR_ADISC) || + !lpfc_is_link_up(vport->phba)) lpfc_nlp_unreg_node(vport, ndlp); } From patchwork Fri Sep 10 23:31:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 509004 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 820F4C433F5 for ; Fri, 10 Sep 2021 23:32:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 65F2061212 for ; Fri, 10 Sep 2021 23:32:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234895AbhIJXdh (ORCPT ); Fri, 10 Sep 2021 19:33:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38590 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234788AbhIJXda (ORCPT ); Fri, 10 Sep 2021 19:33:30 -0400 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8D237C061762 for ; Fri, 10 Sep 2021 16:32:15 -0700 (PDT) Received: by mail-pj1-x102b.google.com with SMTP id lb1-20020a17090b4a4100b001993f863df2so1679105pjb.5 for ; Fri, 10 Sep 2021 16:32:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vkDJTH2aTOmNIXa8p77tBZ2TAUIoZ0mWBrTBpEZwNIs=; b=UJfRogdER0aIHwYmGXwUuLunKcXoyR9yxXVAR/h8kraTuNwLh2WciowSiuGfIsoFXj zGwk0hQOank5bkBMOkqxtp/erLZDJLDHqTE+HHgogpu/1rlVNcd5PCmqyXRxcQTwb6Tj lIfV5vk9x9YQd1ok2YOi0j3nxVwk6/ZznDgWLPVO/LUfDjXv4JuXdwHijLA/BWHH0JGI 2HSyQY/Au7/REm91yYQZUe64vA0I4UC3J4UZ2/75bSB8zhrnZltGUyCFTXjLSI2jh/tZ 4AOOloDJ4W9zLzaTMxX3IfR4nlx8gadwSzl3G39TQeIsH6DB8Q1+7qN4fsNM2V1CzGxd 7FSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vkDJTH2aTOmNIXa8p77tBZ2TAUIoZ0mWBrTBpEZwNIs=; b=ky+kk1ufRn/gZvRCf680SwS+9CyjMv1uRwPUVy56x3h+BdOuPnCICwPsB4NRTPOF7b lr28a9D/XM7zr0gmuydP/j+bdNOzitVln+UTUWANUvlgBHpK7fjvPIgj59kG3wzwI/Oj FwutKBO3jK+dhPngJzIzcLInqSuzBhjnUYWH8VTGURLXka1xir7gY+1pJ4TCedtUtRjh VsWGVdeIQOEnBhS0LpR4lD9iQ1l051bwVWaDmqTYb4OS1Al7lE18sbmaMeH5MzKiNKoT LQvyw+HvuavpznMIxUXYnCfEn4oELA4nXqWG3oF7pITRs0KuXxjMBgHd2rnMd2zwKgXZ BqIw== X-Gm-Message-State: AOAM530vaiae46J+GP+oUIx2u1ZzbRuZFk8rAPrhZrJSGbVHu9YV61Di i8nidREUFUFRTLf65SGhJYa6+wVc3P7rOP6G X-Google-Smtp-Source: ABdhPJyYR/43L7WIRN0vubJcuqDz22Z8En8JoF3SZyPXysCsuWO1MUDMqfeH9eGYsXusPhgzagXsvQ== X-Received: by 2002:a17:90a:9a2:: with SMTP id 31mr95827pjo.58.1631316734825; Fri, 10 Sep 2021 16:32:14 -0700 (PDT) Received: from mail-ash-it-01.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id o15sm11325pfk.143.2021.09.10.16.32.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Sep 2021 16:32:14 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH 06/14] lpfc: Don't remove ndlp on PRLI errors in P2P mode Date: Fri, 10 Sep 2021 16:31:51 -0700 Message-Id: <20210910233159.115896-7-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210910233159.115896-1-jsmart2021@gmail.com> References: <20210910233159.115896-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org In pt-2-pt mode, the initiator does not log into the target after a PRLI error. In pt-2-pt mode, the target responded to the PRLI by sending a LOGO. The LOGO causes all ELS and I/Os to be aborted. This caused the PRLI to fail. The PRLI completion path caused the discovery node to be dropped to avoid being stick in an UNUSED (not logged in) state. As the node was dropped there is no retry of the login and as it is pt-2-pt, there is no RSCN to retrigger discovery. Thus the other end is not seen by the os. Fix by ensuring the discovery node is not dropped if connecting pt-2-pt. This will cause PLOGI to be retried. Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_els.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c index 6c9cb87ef174..c6eae545aabf 100644 --- a/drivers/scsi/lpfc/lpfc_els.c +++ b/drivers/scsi/lpfc/lpfc_els.c @@ -2329,6 +2329,13 @@ lpfc_cmpl_els_prli(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, lpfc_disc_state_machine(vport, ndlp, cmdiocb, NLP_EVT_CMPL_PRLI); + /* + * For P2P topology, retain the node so that PLOGI can be + * attempted on it again. + */ + if (vport->fc_flag & FC_PT2PT) + goto out; + /* As long as this node is not registered with the SCSI * or NVMe transport and no other PRLIs are outstanding, * it is no longer an active node. Otherwise devloss From patchwork Fri Sep 10 23:31:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 509352 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72778C433F5 for ; Fri, 10 Sep 2021 23:32:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5BDD661213 for ; Fri, 10 Sep 2021 23:32:38 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234859AbhIJXdo (ORCPT ); Fri, 10 Sep 2021 19:33:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38592 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234894AbhIJXdb (ORCPT ); Fri, 10 Sep 2021 19:33:31 -0400 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3BAA9C061764 for ; Fri, 10 Sep 2021 16:32:16 -0700 (PDT) Received: by mail-pj1-x1030.google.com with SMTP id dw14so1492602pjb.1 for ; Fri, 10 Sep 2021 16:32:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XgCnLh+b92yUjOifde23K95AaE4qX1Fd7kkLemky5Ds=; b=npYErKmsnd74EXjwqIoO4uIwhNZWrIkrsbwWbou8dNs7J0JpHYG3TqSP4n671UsqbL rmS6ng8HkyBYEfwBYpI9eiPZnGnrG29/rjTlOCpPr6aM8sVlX5tfL6HzBKF8KUCFZcYU sJsBvKJWpJ+00y0XcWkAAj+mAc07roV2f0cwbwHDqVx5m+OnC+D5PDgKlhuNd6baTi71 55h2enbge60T0KwzrtKb0RPeKQdclHkBLPeZJabMfkA7Al7DebEa17oSsj3WXrKGlNMY QsJPMGsXro9s+OBLDivcxx9uENJzrihEFKN57zC737j4ScD9KKwOuZQkYw7ADPj0n3XC PFnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XgCnLh+b92yUjOifde23K95AaE4qX1Fd7kkLemky5Ds=; b=nP519KG4dRsKa5kO+rAo5rqFbUgwwr4F8bi7szP8AvWQmqHeIScS8YWRh0lMFvzFL0 //VHzEEJN7VlGdlnZtoKM0CjqsEa3RWO2asS/HQ3wMHwNnH4vT+yVCi0z8/9yhuS1e/b pBbUrepfgWluwzQuLb6oVspgoOXQ/ZDpMS46UyTYYL8cNu472fOKRKbttRJ1pz2TdVRc JodGEsZKOtinVR1wMB6bPmU9LXI3i6U85xupUfCtm0OBDyWOTLz/plwhgNUTDXzZvwFd uyu2gEjD/1vZkrLGCvmf8BfF3c9zh/u4v1XOP7eKQEMCouz1EKTprnq3Lm6ZnSWl0hHS YTSQ== X-Gm-Message-State: AOAM533zJjYfWqvvHWfLIPqdEEv8mQcMcSJe2Ire2DVcG+oB+l/54tIv q7mhNiZ0B5F5EjwzKvSSzPNxy4DQvUUfn+UF X-Google-Smtp-Source: ABdhPJyfS7Qj1wq79DkZS2pEQgx/foCGmvK1MVqgh0yzQrLKpXzWQJxXJFvr/+pmD4KJD0XPiGxPoA== X-Received: by 2002:a17:90a:e7c8:: with SMTP id kb8mr140912pjb.19.1631316735680; Fri, 10 Sep 2021 16:32:15 -0700 (PDT) Received: from mail-ash-it-01.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id o15sm11325pfk.143.2021.09.10.16.32.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Sep 2021 16:32:15 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH 07/14] lpfc: Fix NVME I/O failover to non-optimized path Date: Fri, 10 Sep 2021 16:31:52 -0700 Message-Id: <20210910233159.115896-8-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210910233159.115896-1-jsmart2021@gmail.com> References: <20210910233159.115896-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Currently, we hold off unregistering with NVME transport layer until GID_FT or ADISC completes upon receipt of RSCN. In the ADISC discovery routine, for nodes not found in the GID_FT response, the nodes are unregistered from the scsi transport but not UNREG_RPI'd. Meaning outstanding WQEs continue to be outstanding and were not failed back to the OS. If an NVMe device, this mean there wasn't initial termination of the I/O's so they could be issued on a different NVME path. Fix by unregistering the RPI so that I/O is cancelled. Fixes: 0614568361b0 ("scsi: lpfc: Delay unregistering from transport until GIDFT or ADISC completes") Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_els.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c index c6eae545aabf..40d166aeb466 100644 --- a/drivers/scsi/lpfc/lpfc_els.c +++ b/drivers/scsi/lpfc/lpfc_els.c @@ -6224,6 +6224,7 @@ lpfc_els_disc_adisc(struct lpfc_vport *vport) * from backend */ lpfc_nlp_unreg_node(vport, ndlp); + lpfc_unreg_rpi(vport, ndlp); continue; } From patchwork Fri Sep 10 23:31:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 509349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76C8AC433EF for ; Fri, 10 Sep 2021 23:33:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6229861215 for ; Fri, 10 Sep 2021 23:33:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234913AbhIJXeG (ORCPT ); Fri, 10 Sep 2021 19:34:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38608 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234871AbhIJXdf (ORCPT ); Fri, 10 Sep 2021 19:33:35 -0400 Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0F9B8C061766; Fri, 10 Sep 2021 16:32:17 -0700 (PDT) Received: by mail-pj1-x102c.google.com with SMTP id lb1-20020a17090b4a4100b001993f863df2so1679151pjb.5; Fri, 10 Sep 2021 16:32:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jGFBDzZt1XuvT2/PvJsUxCaDuknPpPW8lQxF7Hh+uUw=; b=RpJzfrTNh4n7ZEJlgqdUkes6N0sKw2lBi8YI3Df+atU0jAkKhbx0X5ffgmS9DR40Dg nbg9Vv9mOHXfNKRpeRX5IDVMrVNbWs7rJPmIxMSa9L3pSkfmx7kc38/uHmBGSYzm9fUZ MKvn7JTVrPhdbnkpzbWJE8w2f5kggRL03jC/r6KrdZ5MswKlOP7/yGabliBzQa8p9ng6 CCN+OXDGj/Q1TsZ3yzT26xXCGCaH8SmP5GHZAon+FREBSvLHWgo7fhV3Whd6fZm43/b/ BdlHGtFjSwLCozI+xaPhd94PB6n18O5Pg4K+Q3SkQWLNeS/nguurtB/3B91k8xxOmD/v xThA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jGFBDzZt1XuvT2/PvJsUxCaDuknPpPW8lQxF7Hh+uUw=; b=D0SRwUZcRK3OotWsHqw3gonWWoO4DWhC9p4y5G/1UHusWa1y1R9Pj7VZILmB2VuWcn Qc0z8X32WQVhRCpmTusQuv78MjnGFgLQj1+HpFnqjWSU/w/M1A3c3prK+I/JggbUkEZW ehIxfE6M8eVE1vuAaO+xAfGvMZY4R6R8buaCaJt81pbK+waLTH8hSHuyV8ESKgovkOLv uC/bSdESVg2onXZrko87k12gbvzzsmiTgK3V96pmXTwCTjFuJX0wNkpYsN5UpkyJlvOw oGT/HK/vezT2pgDRfVG5NBRIPflhtInBDf+Zu1iv5y3O0R1hZmhigJIj30IjrjCO7dgE NRBg== X-Gm-Message-State: AOAM530BZK6XE5b6JtCwisCENTuL0nrHodRe1C0AaCvQFn1eEqKXl8lF O4HPj94LDr43kfoR46VFvZosz0BbMapf2bNu X-Google-Smtp-Source: ABdhPJxVLxbsQRUxOWJJfi+w0XUCXlK8gBWSIa6H9llvPoBSuXjVdfxF85vTk9U4zgolO4deKoAPvQ== X-Received: by 2002:a17:902:bcc8:b0:13a:8c8:92b8 with SMTP id o8-20020a170902bcc800b0013a08c892b8mr181703pls.88.1631316736343; Fri, 10 Sep 2021 16:32:16 -0700 (PDT) Received: from mail-ash-it-01.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id o15sm11325pfk.143.2021.09.10.16.32.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Sep 2021 16:32:16 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , stable@vger.kernel.org, Justin Tee Subject: [PATCH 08/14] lpfc: Fix FCP I/O flush functionality for TMF routines Date: Fri, 10 Sep 2021 16:31:53 -0700 Message-Id: <20210910233159.115896-9-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210910233159.115896-1-jsmart2021@gmail.com> References: <20210910233159.115896-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org A prior patch inadvertently caused lpfc_sli_sum_iocb to exclude counting of outstanding aborted I/Os and ABORT iocbs. Thus, lpfc_reset_flush_io_context called from any TMF routine does not properly wait to flush all outstanding FCP iocbs leading to a block layer crash on an invalid scsi_cmnd->request pointer. kernel BUG at ../block/blk-core.c:1489! RIP: 0010:blk_requeue_request+0xaf/0xc0 ... Call Trace: __scsi_queue_insert+0x90/0xe0 [scsi_mod] blk_done_softirq+0x7e/0x90 __do_softirq+0xd2/0x280 irq_exit+0xd5/0xe0 do_IRQ+0x4c/0xd0 common_interrupt+0x87/0x87 Fix by separating out the LPFC_IO_FCP, LPFC_IO_ON_TXCMPLQ, LPFC_DRIVER_ABORTED, and CMD_ABORT_XRI_CN || CMD_CLOSE_XRI_CN checks into a new lpfc_sli_validate_fcp_iocb_for_abort routine when determining to build an ABORT iocb. Restore lpfc_reset_flush_io_context functionality by including counting of outstanding aborted iocbs and ABORT iocbs in lpfc_sli_sum_iocb. Fixes: e1364711359f ("scsi: lpfc: Fix illegal memory access on Abort IOCBs") Cc: # v5.12+ Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_sli.c | 101 +++++++++++++++++++++++++++-------- 1 file changed, 78 insertions(+), 23 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index 546c851938bc..e8f6ad484768 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -12485,15 +12485,54 @@ lpfc_sli_hba_iocb_abort(struct lpfc_hba *phba) } /** - * lpfc_sli_validate_fcp_iocb - find commands associated with a vport or LUN + * lpfc_sli_validate_fcp_iocb_for_abort - filter iocbs appropriate for FCP aborts + * @iocbq: Pointer to iocb object. + * @vport: Pointer to driver virtual port object. + * + * This function acts as an iocb filter for functions which abort FCP iocbs. + * + * Return values + * -ENODEV, if a null iocb or vport ptr is encountered + * -EINVAL, if the iocb is not an FCP I/O, not on the TX cmpl queue, premarked as + * driver already started the abort process, or is an abort iocb itself + * 0, passes criteria for aborting the FCP I/O iocb + **/ +static int +lpfc_sli_validate_fcp_iocb_for_abort(struct lpfc_iocbq *iocbq, + struct lpfc_vport *vport) +{ + IOCB_t *icmd = NULL; + + /* No null ptr vports */ + if (!iocbq || iocbq->vport != vport) + return -ENODEV; + + /* iocb must be for FCP IO, already exists on the TX cmpl queue, + * can't be premarked as driver aborted, nor be an ABORT iocb itself + */ + icmd = &iocbq->iocb; + if (!(iocbq->iocb_flag & LPFC_IO_FCP) || + !(iocbq->iocb_flag & LPFC_IO_ON_TXCMPLQ) || + (iocbq->iocb_flag & LPFC_DRIVER_ABORTED) || + (icmd->ulpCommand == CMD_ABORT_XRI_CN || + icmd->ulpCommand == CMD_CLOSE_XRI_CN)) + return -EINVAL; + + return 0; +} + +/** + * lpfc_sli_validate_fcp_iocb - validate commands associated with a SCSI target * @iocbq: Pointer to driver iocb object. * @vport: Pointer to driver virtual port object. * @tgt_id: SCSI ID of the target. * @lun_id: LUN ID of the scsi device. * @ctx_cmd: LPFC_CTX_LUN/LPFC_CTX_TGT/LPFC_CTX_HOST * - * This function acts as an iocb filter for functions which abort or count - * all FCP iocbs pending on a lun/SCSI target/SCSI host. It will return + * This function acts as an iocb filter for validating a lun/SCSI target/SCSI + * host. + * + * It will return * 0 if the filtering criteria is met for the given iocb and will return * 1 if the filtering criteria is not met. * If ctx_cmd == LPFC_CTX_LUN, the function returns 0 only if the @@ -12512,22 +12551,8 @@ lpfc_sli_validate_fcp_iocb(struct lpfc_iocbq *iocbq, struct lpfc_vport *vport, lpfc_ctx_cmd ctx_cmd) { struct lpfc_io_buf *lpfc_cmd; - IOCB_t *icmd = NULL; int rc = 1; - if (!iocbq || iocbq->vport != vport) - return rc; - - if (!(iocbq->iocb_flag & LPFC_IO_FCP) || - !(iocbq->iocb_flag & LPFC_IO_ON_TXCMPLQ) || - iocbq->iocb_flag & LPFC_DRIVER_ABORTED) - return rc; - - icmd = &iocbq->iocb; - if (icmd->ulpCommand == CMD_ABORT_XRI_CN || - icmd->ulpCommand == CMD_CLOSE_XRI_CN) - return rc; - lpfc_cmd = container_of(iocbq, struct lpfc_io_buf, cur_iocbq); if (lpfc_cmd->pCmd == NULL) @@ -12582,17 +12607,33 @@ lpfc_sli_sum_iocb(struct lpfc_vport *vport, uint16_t tgt_id, uint64_t lun_id, { struct lpfc_hba *phba = vport->phba; struct lpfc_iocbq *iocbq; + IOCB_t *icmd = NULL; int sum, i; + unsigned long iflags; - spin_lock_irq(&phba->hbalock); + spin_lock_irqsave(&phba->hbalock, iflags); for (i = 1, sum = 0; i <= phba->sli.last_iotag; i++) { iocbq = phba->sli.iocbq_lookup[i]; - if (lpfc_sli_validate_fcp_iocb (iocbq, vport, tgt_id, lun_id, - ctx_cmd) == 0) + if (!iocbq || iocbq->vport != vport) + continue; + if (!(iocbq->iocb_flag & LPFC_IO_FCP) || + !(iocbq->iocb_flag & LPFC_IO_ON_TXCMPLQ)) + continue; + + /* Include counting outstanding aborts */ + icmd = &iocbq->iocb; + if (icmd->ulpCommand == CMD_ABORT_XRI_CN || + icmd->ulpCommand == CMD_CLOSE_XRI_CN) { + sum++; + continue; + } + + if (lpfc_sli_validate_fcp_iocb(iocbq, vport, tgt_id, lun_id, + ctx_cmd) == 0) sum++; } - spin_unlock_irq(&phba->hbalock); + spin_unlock_irqrestore(&phba->hbalock, iflags); return sum; } @@ -12659,7 +12700,11 @@ lpfc_sli_abort_fcp_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, * * This function sends an abort command for every SCSI command * associated with the given virtual port pending on the ring - * filtered by lpfc_sli_validate_fcp_iocb function. + * filtered by lpfc_sli_validate_fcp_iocb_for_abort and then + * lpfc_sli_validate_fcp_iocb function. The ordering for validation before + * submitting abort iocbs must be lpfc_sli_validate_fcp_iocb_for_abort + * followed by lpfc_sli_validate_fcp_iocb. + * * When abort_cmd == LPFC_CTX_LUN, the function sends abort only to the * FCP iocbs associated with lun specified by tgt_id and lun_id * parameters @@ -12691,6 +12736,9 @@ lpfc_sli_abort_iocb(struct lpfc_vport *vport, u16 tgt_id, u64 lun_id, for (i = 1; i <= phba->sli.last_iotag; i++) { iocbq = phba->sli.iocbq_lookup[i]; + if (lpfc_sli_validate_fcp_iocb_for_abort(iocbq, vport)) + continue; + if (lpfc_sli_validate_fcp_iocb(iocbq, vport, tgt_id, lun_id, abort_cmd) != 0) continue; @@ -12723,7 +12771,11 @@ lpfc_sli_abort_iocb(struct lpfc_vport *vport, u16 tgt_id, u64 lun_id, * * This function sends an abort command for every SCSI command * associated with the given virtual port pending on the ring - * filtered by lpfc_sli_validate_fcp_iocb function. + * filtered by lpfc_sli_validate_fcp_iocb_for_abort and then + * lpfc_sli_validate_fcp_iocb function. The ordering for validation before + * submitting abort iocbs must be lpfc_sli_validate_fcp_iocb_for_abort + * followed by lpfc_sli_validate_fcp_iocb. + * * When taskmgmt_cmd == LPFC_CTX_LUN, the function sends abort only to the * FCP iocbs associated with lun specified by tgt_id and lun_id * parameters @@ -12761,6 +12813,9 @@ lpfc_sli_abort_taskmgmt(struct lpfc_vport *vport, struct lpfc_sli_ring *pring, for (i = 1; i <= phba->sli.last_iotag; i++) { iocbq = phba->sli.iocbq_lookup[i]; + if (lpfc_sli_validate_fcp_iocb_for_abort(iocbq, vport)) + continue; + if (lpfc_sli_validate_fcp_iocb(iocbq, vport, tgt_id, lun_id, cmd) != 0) continue; From patchwork Fri Sep 10 23:31:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 509002 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53AADC433EF for ; Fri, 10 Sep 2021 23:32:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 351F461212 for ; Fri, 10 Sep 2021 23:32:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234809AbhIJXdw (ORCPT ); Fri, 10 Sep 2021 19:33:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234905AbhIJXdf (ORCPT ); Fri, 10 Sep 2021 19:33:35 -0400 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F347BC061767 for ; Fri, 10 Sep 2021 16:32:17 -0700 (PDT) Received: by mail-pj1-x1031.google.com with SMTP id mi6-20020a17090b4b4600b00199280a31cbso2049490pjb.0 for ; Fri, 10 Sep 2021 16:32:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Yq2JC2yQmNtz50ZXurmg9MwpHzN0kcLpRz8Hcr+wqik=; b=B28gtTFiCdp7jvbEjAjQtGTORK11tObxC3mTXdy6Av0lCQnpMorzHynAynu/Luj6I/ DsVG4Vd5vFUPGXu+IM6XcQr2y/b2mK576iRkOzVri6d4pddj8k3xGkjYFZBl8WXqwIab UdSPF9sgtd0+bP9/hwt1qypllnBFtUpSxfDGaT9YKjWLaTDSEk4x4ux1WJBgu7LCSc1b 3zDUGVQaBAPFTtGF1dXF944nwS/SzBk/grvti1eDeZdHU4VUaKmF7gmM6oDHVrIq+2Ag kUeRp4r3EhKiZlfc2oRsEPpKEoBZq3sG+s8k37avJZYAL8S+yrQu+LkKpIQEmijSsVOS c3BQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Yq2JC2yQmNtz50ZXurmg9MwpHzN0kcLpRz8Hcr+wqik=; b=0qtWC0xSnBZGMpa1FIeXNY+oulIx8zoj9W1DWyPkuvFolMikGHeFVtj/vym7cAFsPG zq0szL0Bamm5MVDyBjbH8K81D65WSj7uj1NdFZ+m2QiA+LpUQkfGgYsCrj9ryhUywirn a0AeFq5L6Y5psZ/UIBfAuHzFQjtkLhOh3Ssle7Cop4QGSadb5rKbcfYiFWjpFYosZQJR gsIZhUX3Y7slaGs8/qjPvn92yGI0Hmpq650d+xJ07gnR7EedwaELLjZpFJEFBD6mq0mP 0zRuPBUojtpt/knipuT+w2zw5GZ4mr/Wdp1LLuvX0D7o6DBhKJsvQBd8g7TA0r2e/6PL ThgA== X-Gm-Message-State: AOAM531w9cm55T7gi9aj7ewwadrRNP/1JmH74meXuspQu+5Rbdnh7D09 a+Sf2UXqX3lsIVTH37AR82wmCctXgBAfba+7 X-Google-Smtp-Source: ABdhPJyqosrQ9jIv/tt968yUPBjYDvle+jlBCJdqxq73iUzdRcftiK9LrTv7UiAqSTn52kZI0o5n1w== X-Received: by 2002:a17:90a:e009:: with SMTP id u9mr78305pjy.218.1631316737125; Fri, 10 Sep 2021 16:32:17 -0700 (PDT) Received: from mail-ash-it-01.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id o15sm11325pfk.143.2021.09.10.16.32.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Sep 2021 16:32:16 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH 09/14] lpfc: Fix EEH support for NVME I/O Date: Fri, 10 Sep 2021 16:31:54 -0700 Message-Id: <20210910233159.115896-10-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210910233159.115896-1-jsmart2021@gmail.com> References: <20210910233159.115896-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Injecting errors on the pci-slot while the driver is handling nvme io will cause crashes and hangs. There are several rather difficult scenarios occurring. The main issue is that the adapter can report a pci error before or simultaneously to the pci subsystem reporting the error. Both paths have different entry points and currently, there is no interlock between them. Thus multiple teardown paths are competing and all heck breaks loose. Complicating things is the nvme path. To a large degree, i/o was able to be shutdown for a full FC port on the scsi stack. But on NVMe, there isn't a similar call. At best, it works on a per-controller basis, but even at the controller level, it's a controller "reset" call. All of which means i/o is still flowing on different cpus with reset paths expecting hw access (mailbox commands) to execute properly. The following modifications are made in this patch: - A new flag is set in PCI error entrypoints so the driver can track being called by that path. - A interlock is added in the sli hw error path and the pci error path such that only one of the paths proceeds with the teardown logic. - RPI cleanup is patched such that rpis are marked unregistered w/o mbx cmds in cases of hw error. - If entering the sli port re-init calls, a case where sli error teardown was quick and beat the pci calls now reporting error, check whether the sli port is still live on the pci bus before - in the pci reset code to bring the adapter back, recheck the irq settings. Different checks for sli3 vs sli4. - in i/o completions, that may be called as part of the cleanup or underway just before the hw error, check the state of the adapter. If in error, short cut handling that would expect further adapter completions as the hw error won't be sending them. - in routines waiting on i/o completions, which may have been in progress prior to the hw error, detect the device is being torn down and abort from their waits and just give up. This points to a larger issue in the driver on ref-counting for data structures, as it doesn't have ref-counting on q and port structures. We'll do this fix for now as it would be a major rework to be done differently. - fix the nvme cleanup to simulate nvme i/o completions if i/o is being failed back due to hw error. - in io buf allocation, done at the start of new ios, check hw state and fail if hw error. Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc.h | 1 + drivers/scsi/lpfc/lpfc_hbadisc.c | 1 + drivers/scsi/lpfc/lpfc_init.c | 31 +++++++++++++++- drivers/scsi/lpfc/lpfc_nvme.c | 53 ++++++++++++++++++++++++++- drivers/scsi/lpfc/lpfc_scsi.c | 63 +++++++++++++++++++++----------- drivers/scsi/lpfc/lpfc_sli.c | 31 +++++++++++++++- drivers/scsi/lpfc/lpfc_sli4.h | 2 + 7 files changed, 154 insertions(+), 28 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h index befeb7c34290..2f6b81bc2451 100644 --- a/drivers/scsi/lpfc/lpfc.h +++ b/drivers/scsi/lpfc/lpfc.h @@ -1028,6 +1028,7 @@ struct lpfc_hba { * Firmware supports Forced Link Speed * capability */ +#define HBA_PCI_ERR 0x80000 /* The PCI slot is offline */ #define HBA_FLOGI_ISSUED 0x100000 /* FLOGI was issued */ #define HBA_CGN_RSVD1 0x200000 /* Reserved CGN flag */ #define HBA_CGN_DAY_WRAP 0x400000 /* HBA Congestion info day wraps */ diff --git a/drivers/scsi/lpfc/lpfc_hbadisc.c b/drivers/scsi/lpfc/lpfc_hbadisc.c index 12abfc027a67..0b1e1cc00e01 100644 --- a/drivers/scsi/lpfc/lpfc_hbadisc.c +++ b/drivers/scsi/lpfc/lpfc_hbadisc.c @@ -5250,6 +5250,7 @@ lpfc_unreg_rpi(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) rc = lpfc_sli_issue_mbox(phba, mbox, MBX_NOWAIT); if (rc == MBX_NOT_FINISHED) { + ndlp->nlp_flag &= ~NLP_UNREG_INP; mempool_free(mbox, phba->mbox_mem_pool); acc_plogi = 1; } diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c index 6e742d31258d..ea57151221cd 100644 --- a/drivers/scsi/lpfc/lpfc_init.c +++ b/drivers/scsi/lpfc/lpfc_init.c @@ -1606,6 +1606,11 @@ void lpfc_sli4_offline_eratt(struct lpfc_hba *phba) { spin_lock_irq(&phba->hbalock); + if (phba->link_state == LPFC_HBA_ERROR && + phba->hba_flag & HBA_PCI_ERR) { + spin_unlock_irq(&phba->hbalock); + return; + } phba->link_state = LPFC_HBA_ERROR; spin_unlock_irq(&phba->hbalock); @@ -1945,7 +1950,6 @@ lpfc_handle_eratt_s4(struct lpfc_hba *phba) if (pci_channel_offline(phba->pcidev)) { lpfc_printf_log(phba, KERN_ERR, LOG_TRACE_EVENT, "3166 pci channel is offline\n"); - lpfc_sli4_offline_eratt(phba); return; } @@ -3643,6 +3647,7 @@ lpfc_offline_prep(struct lpfc_hba *phba, int mbx_action) struct lpfc_vport **vports; struct Scsi_Host *shost; int i; + int offline = 0; if (vport->fc_flag & FC_OFFLINE_MODE) return; @@ -3651,6 +3656,8 @@ lpfc_offline_prep(struct lpfc_hba *phba, int mbx_action) lpfc_linkdown(phba); + offline = pci_channel_offline(phba->pcidev); + /* Issue an unreg_login to all nodes on all vports */ vports = lpfc_create_vport_work_array(phba); if (vports != NULL) { @@ -3673,7 +3680,14 @@ lpfc_offline_prep(struct lpfc_hba *phba, int mbx_action) ndlp->nlp_flag &= ~NLP_NPR_ADISC; spin_unlock_irq(&ndlp->lock); - lpfc_unreg_rpi(vports[i], ndlp); + if (offline) { + spin_lock_irq(&ndlp->lock); + ndlp->nlp_flag &= ~(NLP_UNREG_INP | + NLP_RPI_REGISTERED); + spin_unlock_irq(&ndlp->lock); + } else { + lpfc_unreg_rpi(vports[i], ndlp); + } /* * Whenever an SLI4 port goes offline, free the * RPI. Get a new RPI when the adapter port @@ -14080,6 +14094,10 @@ lpfc_pci_resume_one_s3(struct device *dev_d) return error; } + /* Init cpu_map array */ + lpfc_cpu_map_array_init(phba); + /* Init hba_eq_hdl array */ + lpfc_hba_eq_hdl_array_init(phba); /* Configure and enable interrupt */ intr_mode = lpfc_sli_enable_intr(phba, phba->intr_mode); if (intr_mode == LPFC_INTR_ERROR) { @@ -15032,14 +15050,17 @@ lpfc_io_error_detected_s4(struct pci_dev *pdev, pci_channel_state_t state) lpfc_sli4_prep_dev_for_recover(phba); return PCI_ERS_RESULT_CAN_RECOVER; case pci_channel_io_frozen: + phba->hba_flag |= HBA_PCI_ERR; /* Fatal error, prepare for slot reset */ lpfc_sli4_prep_dev_for_reset(phba); return PCI_ERS_RESULT_NEED_RESET; case pci_channel_io_perm_failure: + phba->hba_flag |= HBA_PCI_ERR; /* Permanent failure, prepare for device down */ lpfc_sli4_prep_dev_for_perm_failure(phba); return PCI_ERS_RESULT_DISCONNECT; default: + phba->hba_flag |= HBA_PCI_ERR; /* Unknown state, prepare and request slot reset */ lpfc_printf_log(phba, KERN_ERR, LOG_TRACE_EVENT, "2825 Unknown PCI error state: x%x\n", state); @@ -15083,6 +15104,7 @@ lpfc_io_slot_reset_s4(struct pci_dev *pdev) pci_restore_state(pdev); + phba->hba_flag &= ~HBA_PCI_ERR; /* * As the new kernel behavior of pci_restore_state() API call clears * device saved_state flag, need to save the restored state again. @@ -15105,6 +15127,7 @@ lpfc_io_slot_reset_s4(struct pci_dev *pdev) return PCI_ERS_RESULT_DISCONNECT; } else phba->intr_mode = intr_mode; + lpfc_cpu_affinity_check(phba, phba->cfg_irq_chann); /* Log the current active interrupt mode */ lpfc_log_intr_mode(phba, phba->intr_mode); @@ -15306,6 +15329,10 @@ lpfc_io_error_detected(struct pci_dev *pdev, pci_channel_state_t state) struct lpfc_hba *phba = ((struct lpfc_vport *)shost->hostdata)->phba; pci_ers_result_t rc = PCI_ERS_RESULT_DISCONNECT; + if (phba->link_state == LPFC_HBA_ERROR && + phba->hba_flag & HBA_IOQ_FLUSH) + return PCI_ERS_RESULT_NEED_RESET; + switch (phba->pci_dev_grp) { case LPFC_PCI_DEV_LP: rc = lpfc_io_error_detected_s3(pdev, state); diff --git a/drivers/scsi/lpfc/lpfc_nvme.c b/drivers/scsi/lpfc/lpfc_nvme.c index bd88477f9b82..33266e1b24ab 100644 --- a/drivers/scsi/lpfc/lpfc_nvme.c +++ b/drivers/scsi/lpfc/lpfc_nvme.c @@ -937,6 +937,7 @@ lpfc_nvme_io_cmd_wqe_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pwqeIn, #ifdef CONFIG_SCSI_LPFC_DEBUG_FS int cpu; #endif + int offline = 0; /* Sanity check on return of outstanding command */ if (!lpfc_ncmd) { @@ -1098,11 +1099,12 @@ lpfc_nvme_io_cmd_wqe_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pwqeIn, nCmd->transferred_length = 0; nCmd->rcv_rsplen = 0; nCmd->status = NVME_SC_INTERNAL; + offline = pci_channel_offline(vport->phba->pcidev); } } /* pick up SLI4 exhange busy condition */ - if (bf_get(lpfc_wcqe_c_xb, wcqe)) + if (bf_get(lpfc_wcqe_c_xb, wcqe) && !offline) lpfc_ncmd->flags |= LPFC_SBUF_XBUSY; else lpfc_ncmd->flags &= ~LPFC_SBUF_XBUSY; @@ -2169,6 +2171,10 @@ lpfc_nvme_lport_unreg_wait(struct lpfc_vport *vport, abts_nvme = 0; for (i = 0; i < phba->cfg_hdw_queue; i++) { qp = &phba->sli4_hba.hdwq[i]; + if (!vport || !vport->localport || + !qp || !qp->io_wq) + return; + pring = qp->io_wq->pring; if (!pring) continue; @@ -2176,6 +2182,10 @@ lpfc_nvme_lport_unreg_wait(struct lpfc_vport *vport, abts_scsi += qp->abts_scsi_io_bufs; abts_nvme += qp->abts_nvme_io_bufs; } + if (!vport || !vport->localport || + vport->phba->hba_flag & HBA_PCI_ERR) + return; + lpfc_printf_vlog(vport, KERN_ERR, LOG_TRACE_EVENT, "6176 Lport x%px Localport x%px wait " "timed out. Pending %d [%d:%d]. " @@ -2215,6 +2225,8 @@ lpfc_nvme_destroy_localport(struct lpfc_vport *vport) return; localport = vport->localport; + if (!localport) + return; lport = (struct lpfc_nvme_lport *)localport->private; lpfc_printf_vlog(vport, KERN_INFO, LOG_NVME, @@ -2531,7 +2543,8 @@ lpfc_nvme_unregister_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) * return values is ignored. The upcall is a courtesy to the * transport. */ - if (vport->load_flag & FC_UNLOADING) + if (vport->load_flag & FC_UNLOADING || + unlikely(vport->phba->hba_flag & HBA_PCI_ERR)) (void)nvme_fc_set_remoteport_devloss(remoteport, 0); ret = nvme_fc_unregister_remoteport(remoteport); @@ -2559,6 +2572,42 @@ lpfc_nvme_unregister_port(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp) vport->localport, ndlp->rport, ndlp->nlp_DID); } +/** + * lpfc_sli4_nvme_pci_offline_aborted - Fast-path process of NVME xri abort + * @phba: pointer to lpfc hba data structure. + * @lpfc_ncmd: The nvme job structure for the request being aborted. + * + * This routine is invoked by the worker thread to process a SLI4 fast-path + * NVME aborted xri. Aborted NVME IO commands are completed to the transport + * here. + **/ +void +lpfc_sli4_nvme_pci_offline_aborted(struct lpfc_hba *phba, + struct lpfc_io_buf *lpfc_ncmd) +{ + struct nvmefc_fcp_req *nvme_cmd = NULL; + + lpfc_printf_log(phba, KERN_INFO, LOG_NVME_ABTS, + "6533 %s nvme_cmd %p tag x%x abort complete and " + "xri released\n", __func__, + lpfc_ncmd->nvmeCmd, + lpfc_ncmd->cur_iocbq.iotag); + + /* Aborted NVME commands are required to not complete + * before the abort exchange command fully completes. + * Once completed, it is available via the put list. + */ + if (lpfc_ncmd->nvmeCmd) { + nvme_cmd = lpfc_ncmd->nvmeCmd; + nvme_cmd->transferred_length = 0; + nvme_cmd->rcv_rsplen = 0; + nvme_cmd->status = NVME_SC_INTERNAL; + nvme_cmd->done(nvme_cmd); + lpfc_ncmd->nvmeCmd = NULL; + } + lpfc_release_nvme_buf(phba, lpfc_ncmd); +} + /** * lpfc_sli4_nvme_xri_aborted - Fast-path process of NVME xri abort * @phba: pointer to lpfc hba data structure. diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c index 0fde1e874c7a..b70f71b5c1f7 100644 --- a/drivers/scsi/lpfc/lpfc_scsi.c +++ b/drivers/scsi/lpfc/lpfc_scsi.c @@ -493,8 +493,8 @@ void lpfc_sli4_io_xri_aborted(struct lpfc_hba *phba, struct sli4_wcqe_xri_aborted *axri, int idx) { - uint16_t xri = bf_get(lpfc_wcqe_xa_xri, axri); - uint16_t rxid = bf_get(lpfc_wcqe_xa_remote_xid, axri); + u16 xri = 0; + u16 rxid = 0; struct lpfc_io_buf *psb, *next_psb; struct lpfc_sli4_hdw_queue *qp; unsigned long iflag = 0; @@ -504,15 +504,22 @@ lpfc_sli4_io_xri_aborted(struct lpfc_hba *phba, int rrq_empty = 0; struct lpfc_sli_ring *pring = phba->sli4_hba.els_wq->pring; struct scsi_cmnd *cmd; + int offline = 0; if (!(phba->cfg_enable_fc4_type & LPFC_ENABLE_FCP)) return; - + offline = pci_channel_offline(phba->pcidev); + if (!offline) { + xri = bf_get(lpfc_wcqe_xa_xri, axri); + rxid = bf_get(lpfc_wcqe_xa_remote_xid, axri); + } qp = &phba->sli4_hba.hdwq[idx]; spin_lock_irqsave(&phba->hbalock, iflag); spin_lock(&qp->abts_io_buf_list_lock); list_for_each_entry_safe(psb, next_psb, &qp->lpfc_abts_io_buf_list, list) { + if (offline) + xri = psb->cur_iocbq.sli4_xritag; if (psb->cur_iocbq.sli4_xritag == xri) { list_del_init(&psb->list); psb->flags &= ~LPFC_SBUF_XBUSY; @@ -521,8 +528,15 @@ lpfc_sli4_io_xri_aborted(struct lpfc_hba *phba, qp->abts_nvme_io_bufs--; spin_unlock(&qp->abts_io_buf_list_lock); spin_unlock_irqrestore(&phba->hbalock, iflag); - lpfc_sli4_nvme_xri_aborted(phba, axri, psb); - return; + if (!offline) { + lpfc_sli4_nvme_xri_aborted(phba, axri, + psb); + return; + } + lpfc_sli4_nvme_pci_offline_aborted(phba, psb); + spin_lock_irqsave(&phba->hbalock, iflag); + spin_lock(&qp->abts_io_buf_list_lock); + continue; } qp->abts_scsi_io_bufs--; spin_unlock(&qp->abts_io_buf_list_lock); @@ -534,13 +548,13 @@ lpfc_sli4_io_xri_aborted(struct lpfc_hba *phba, rrq_empty = list_empty(&phba->active_rrq_list); spin_unlock_irqrestore(&phba->hbalock, iflag); - if (ndlp) { + if (ndlp && !offline) { lpfc_set_rrq_active(phba, ndlp, psb->cur_iocbq.sli4_lxritag, rxid, 1); lpfc_sli4_abts_err_handler(phba, ndlp, axri); } - if (phba->cfg_fcp_wait_abts_rsp) { + if (phba->cfg_fcp_wait_abts_rsp || offline) { spin_lock_irqsave(&psb->buf_lock, iflag); cmd = psb->pCmd; psb->pCmd = NULL; @@ -567,25 +581,30 @@ lpfc_sli4_io_xri_aborted(struct lpfc_hba *phba, lpfc_release_scsi_buf_s4(phba, psb); if (rrq_empty) lpfc_worker_wake_up(phba); - return; + if (!offline) + return; + spin_lock_irqsave(&phba->hbalock, iflag); + spin_lock(&qp->abts_io_buf_list_lock); + continue; } } spin_unlock(&qp->abts_io_buf_list_lock); - for (i = 1; i <= phba->sli.last_iotag; i++) { - iocbq = phba->sli.iocbq_lookup[i]; - - if (!(iocbq->iocb_flag & LPFC_IO_FCP) || - (iocbq->iocb_flag & LPFC_IO_LIBDFC)) - continue; - if (iocbq->sli4_xritag != xri) - continue; - psb = container_of(iocbq, struct lpfc_io_buf, cur_iocbq); - psb->flags &= ~LPFC_SBUF_XBUSY; - spin_unlock_irqrestore(&phba->hbalock, iflag); - if (!list_empty(&pring->txq)) - lpfc_worker_wake_up(phba); - return; + if (!offline) { + for (i = 1; i <= phba->sli.last_iotag; i++) { + iocbq = phba->sli.iocbq_lookup[i]; + if (!(iocbq->iocb_flag & LPFC_IO_FCP) || + (iocbq->iocb_flag & LPFC_IO_LIBDFC)) + continue; + if (iocbq->sli4_xritag != xri) + continue; + psb = container_of(iocbq, struct lpfc_io_buf, cur_iocbq); + psb->flags &= ~LPFC_SBUF_XBUSY; + spin_unlock_irqrestore(&phba->hbalock, iflag); + if (!list_empty(&pring->txq)) + lpfc_worker_wake_up(phba); + return; + } } spin_unlock_irqrestore(&phba->hbalock, iflag); } diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index e8f6ad484768..651e6ee64e88 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -1404,7 +1404,8 @@ __lpfc_sli_release_iocbq_s4(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq) } if ((iocbq->iocb_flag & LPFC_EXCHANGE_BUSY) && - (sglq->state != SGL_XRI_ABORTED)) { + (!(unlikely(pci_channel_offline(phba->pcidev)))) && + sglq->state != SGL_XRI_ABORTED) { spin_lock_irqsave(&phba->sli4_hba.sgl_list_lock, iflag); @@ -4583,10 +4584,12 @@ lpfc_sli_flush_io_rings(struct lpfc_hba *phba) lpfc_sli_cancel_iocbs(phba, &txq, IOSTAT_LOCAL_REJECT, IOERR_SLI_DOWN); - /* Flush the txcmpq */ + /* Flush the txcmplq */ lpfc_sli_cancel_iocbs(phba, &txcmplq, IOSTAT_LOCAL_REJECT, IOERR_SLI_DOWN); + if (unlikely(pci_channel_offline(phba->pcidev))) + lpfc_sli4_io_xri_aborted(phba, NULL, 0); } } else { pring = &psli->sli3_ring[LPFC_FCP_RING]; @@ -22019,8 +22022,26 @@ lpfc_get_io_buf_from_multixri_pools(struct lpfc_hba *phba, qp = &phba->sli4_hba.hdwq[hwqid]; lpfc_ncmd = NULL; + if (!qp) { + lpfc_printf_log(phba, KERN_INFO, + LOG_SLI | LOG_NVME_ABTS | LOG_FCP, + "5556 NULL qp for hwqid x%x\n", hwqid); + return lpfc_ncmd; + } multixri_pool = qp->p_multixri_pool; + if (!multixri_pool) { + lpfc_printf_log(phba, KERN_INFO, + LOG_SLI | LOG_NVME_ABTS | LOG_FCP, + "5557 NULL multixri for hwqid x%x\n", hwqid); + return lpfc_ncmd; + } pvt_pool = &multixri_pool->pvt_pool; + if (!pvt_pool) { + lpfc_printf_log(phba, KERN_INFO, + LOG_SLI | LOG_NVME_ABTS | LOG_FCP, + "5558 NULL pvt_pool for hwqid x%x\n", hwqid); + return lpfc_ncmd; + } multixri_pool->io_req_count++; /* If pvt_pool is empty, move some XRIs from public to private pool */ @@ -22096,6 +22117,12 @@ struct lpfc_io_buf *lpfc_get_io_buf(struct lpfc_hba *phba, qp = &phba->sli4_hba.hdwq[hwqid]; lpfc_cmd = NULL; + if (!qp) { + lpfc_printf_log(phba, KERN_WARNING, + LOG_SLI | LOG_NVME_ABTS | LOG_FCP, + "5555 NULL qp for hwqid x%x\n", hwqid); + return lpfc_cmd; + } if (phba->cfg_xri_rebalancing) lpfc_cmd = lpfc_get_io_buf_from_multixri_pools( diff --git a/drivers/scsi/lpfc/lpfc_sli4.h b/drivers/scsi/lpfc/lpfc_sli4.h index 99c5d1e4da5e..5962cf508842 100644 --- a/drivers/scsi/lpfc/lpfc_sli4.h +++ b/drivers/scsi/lpfc/lpfc_sli4.h @@ -1116,6 +1116,8 @@ void lpfc_sli4_fcf_redisc_event_proc(struct lpfc_hba *); int lpfc_sli4_resume_rpi(struct lpfc_nodelist *, void (*)(struct lpfc_hba *, LPFC_MBOXQ_t *), void *); void lpfc_sli4_els_xri_abort_event_proc(struct lpfc_hba *phba); +void lpfc_sli4_nvme_pci_offline_aborted(struct lpfc_hba *phba, + struct lpfc_io_buf *lpfc_ncmd); void lpfc_sli4_nvme_xri_aborted(struct lpfc_hba *phba, struct sli4_wcqe_xri_aborted *axri, struct lpfc_io_buf *lpfc_ncmd); From patchwork Fri Sep 10 23:31:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 509003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id AEDBEC433FE for ; Fri, 10 Sep 2021 23:32:39 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9717161206 for ; Fri, 10 Sep 2021 23:32:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234894AbhIJXdt (ORCPT ); Fri, 10 Sep 2021 19:33:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38612 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234873AbhIJXdf (ORCPT ); Fri, 10 Sep 2021 19:33:35 -0400 Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 953E8C061768 for ; Fri, 10 Sep 2021 16:32:18 -0700 (PDT) Received: by mail-pg1-x52b.google.com with SMTP id k24so3212799pgh.8 for ; Fri, 10 Sep 2021 16:32:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6RaOTeQITjjS5RD7oWNsJ4O3kzDKzTz3KbH8dZABJIY=; b=Ik8AgBIACuEKCe7WnQ9yKdGUhKWYrUwtYTs5NYRpWTMrM4Jz6o9ZUTViwMRgDiWQRX Tcejy36I2tJytddvDvA8mTc8BazZOcvqNt+wZPJZaMmB4HRypIxW6zIRFd5zMJXFpqMr ME9acUvMFyqkL4WgD5+hbaagzNsxstq2msgEYX1QOr5e8GvgSF4dd5US6BTeqC3dJvbn P4B2jgdqqTmyeexvykXVdnCe8BMSn8myB7IxdULmVf6oUA7yiyjNFgsEKKbuH/hyZ8Qt ZvRMKxOXyaA5dDDT69Nf5Ad41rF1RRfHpBMQrciLtcvDuBEopQ8/R7+KSxfcq2exoUS4 VEtQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6RaOTeQITjjS5RD7oWNsJ4O3kzDKzTz3KbH8dZABJIY=; b=2ssoDPOgzmPkUt7S/zyacPClWWf6Zuy4pA7zSTaC90qZmut24S7beaNHvrVA8GeHzM NoIqz1oij/gcgxF5N9ebdnMW7bUniZI8vM4E/mlNvCm0wkIS0xeYMUnO+aPT9U0L4+0n P9Jz+imrX+EsolpKerXRknSD8Fd2QVKG+3G3Yv5llWzlXGvZp32RYviG9PhyUpv0+ZxI agq3agEZrXVqdNj5GbXQhIsE9efDuHAvLuqvxguB+6xWPPAv6xPtLU5Bx3H9SHiKQoKT tBU5i/IUC3iIX6a9Su6+6seReveNrvv1rU+cSaj6xXCM+7B//VUxq6qcpXUoKTn1F/xt hrDw== X-Gm-Message-State: AOAM531dr4fypTXpxEYcZ4nvPbV77TeG+oyQL6BzaDQy6bwM6cvtsgM+ wOATlGyTalQIEgYti6QfTz8///XZSQ75gqoN X-Google-Smtp-Source: ABdhPJwh+fFThjBM4ijfHv7/95ZDGCrCldyyhqTjMZIUKE5P/Acu+ipbxzaIvCxCgzYZyQmIMKuXDg== X-Received: by 2002:aa7:9570:0:b0:437:3bea:4366 with SMTP id x16-20020aa79570000000b004373bea4366mr84747pfq.16.1631316737927; Fri, 10 Sep 2021 16:32:17 -0700 (PDT) Received: from mail-ash-it-01.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id o15sm11325pfk.143.2021.09.10.16.32.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Sep 2021 16:32:17 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH 10/14] lpfc: Adjust bytes received vales during cmf timer interval Date: Fri, 10 Sep 2021 16:31:55 -0700 Message-Id: <20210910233159.115896-11-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210910233159.115896-1-jsmart2021@gmail.com> References: <20210910233159.115896-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The newly added congestion mgmt framework is seeing unexpected congestion FPINs and signals. In analysis, time values given to the adapter are not at hard time intervals. Thus the drift vs the transfer count seen is affecting how the framework manages things. Adjust counters to cover the drift. Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_init.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c index ea57151221cd..4637531f3b02 100644 --- a/drivers/scsi/lpfc/lpfc_init.c +++ b/drivers/scsi/lpfc/lpfc_init.c @@ -5876,7 +5876,7 @@ lpfc_cmf_timer(struct hrtimer *timer) uint32_t io_cnt; uint32_t head, tail; uint32_t busy, max_read; - uint64_t total, rcv, lat, mbpi; + uint64_t total, rcv, lat, mbpi, extra; int timer_interval = LPFC_CMF_INTERVAL; uint32_t ms; struct lpfc_cgn_stat *cgs; @@ -5943,7 +5943,19 @@ lpfc_cmf_timer(struct hrtimer *timer) phba->hba_flag & HBA_SETUP) { mbpi = phba->cmf_last_sync_bw; phba->cmf_last_sync_bw = 0; - lpfc_issue_cmf_sync_wqe(phba, LPFC_CMF_INTERVAL, total); + extra = 0; + + /* Calculate any extra bytes needed to account for the + * timer accuracy. If we are less than LPFC_CMF_INTERVAL + * add an extra 3% slop factor, equal to LPFC_CMF_INTERVAL + * add an extra 2%. The goal is to equalize total with a + * time > LPFC_CMF_INTERVAL or <= LPFC_CMF_INTERVAL + 1 + */ + if (ms == LPFC_CMF_INTERVAL) + extra = div_u64(total, 50); + else if (ms < LPFC_CMF_INTERVAL) + extra = div_u64(total, 33); + lpfc_issue_cmf_sync_wqe(phba, LPFC_CMF_INTERVAL, total + extra); } else { /* For Monitor mode or link down we want mbpi * to be the full link speed From patchwork Fri Sep 10 23:31:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 509000 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84CCBC433FE for ; Fri, 10 Sep 2021 23:33:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6F10F61216 for ; Fri, 10 Sep 2021 23:33:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234920AbhIJXeH (ORCPT ); Fri, 10 Sep 2021 19:34:07 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38614 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234907AbhIJXdf (ORCPT ); Fri, 10 Sep 2021 19:33:35 -0400 Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B7683C061574 for ; Fri, 10 Sep 2021 16:32:19 -0700 (PDT) Received: by mail-pg1-x52c.google.com with SMTP id e7so3240891pgk.2 for ; Fri, 10 Sep 2021 16:32:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=s1Azg02AtHBirXEqyTq6HoZbHpnb4Qp9PyPKAwI31As=; b=MWsTc1RUsGrMXWPSgxKwKRcDOc8SkIqBAiU18WOKXL4B7xipyZ/A2BBf3NFrhis0bx 6oUSanDYCX+0+rRCjdKfRQXTq4mH0hOadwfRBaCV5LXDFySAHMWxJTQNh917opP2v52G 8xXCV30jGfI/RwfbV+tpU4SFSkmUcHT0q7NzjZdGDOVKayp35yrIQF22Hrmni3sBVURX zgnMLgNddg2jp5mazWsa8/VDqFBApuBpmadHL8gUR/8mCgRcuk9SSs5Fm+3wMDe5YP5j uq2ACTJ11EyPKKYkzAE95KX9XDqLSaCZk7wfGNy/uVg3Ra2RQecoV/m1jFAP9uM+2dNb uslQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=s1Azg02AtHBirXEqyTq6HoZbHpnb4Qp9PyPKAwI31As=; b=F3AG4NFHtdrbbQP3osHoJbfUvPQz14F+oPN2rlQocKeRAHqFO1TR/uWa3UZvkQaA5t WcUg0d5wUSgl/8q3t4Fb4l74WO3X64ujWNkJF5UlXCLxCfLkGFokoxHSiH8EO2OiD8pj sEtSeyRaqLGNWhbn2Si2IDOuxpmp51+oo7zGoS6LPhlyRqKoo6WxDkp9SV+QWEzK2nEB 8rAOCI+FjxAhtI+zdJ+0PBmwkQaKZIGYjKfBaAi9ZldtZv9hUQ1zCAyPkwRVSsQEPZiM u/zA67Ki0ofLKDdsKY9vwwns/+AC/xTWsUrMb1ueMrvF6Xzx4hImfArrGv6uZYnNjMb6 OCHw== X-Gm-Message-State: AOAM532GqkVEoSzsPGOSYqOEUo/Qtuxylivx46KKhE2uwl+lLuSNGcq2 cNZfSAyuOJy7wWg0RGy/Js+7yatMPTywiWlw X-Google-Smtp-Source: ABdhPJxDtyvv8PrxyuwdlbdFxPM5SWXt0MkKwTz/kkhizA66gBIJg3P0EhhphnBZxMgdH2Dj/So0nA== X-Received: by 2002:a62:ab04:0:b0:405:c1c0:543c with SMTP id p4-20020a62ab04000000b00405c1c0543cmr10511666pff.36.1631316738901; Fri, 10 Sep 2021 16:32:18 -0700 (PDT) Received: from mail-ash-it-01.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id o15sm11325pfk.143.2021.09.10.16.32.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Sep 2021 16:32:18 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH 11/14] lpfc: Fix I/O block after enabling managed congestion mode Date: Fri, 10 Sep 2021 16:31:56 -0700 Message-Id: <20210910233159.115896-12-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210910233159.115896-1-jsmart2021@gmail.com> References: <20210910233159.115896-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org If the congestion management framework dynamically enables, it may do so while I/O is in flight. The updates of cmf info due to inflight I/O completing may happen before values have been initialized. Fix by ensure cmf_max_bytes_per_interval is initialized when checking bandwidth utilization for SCSI layer blocking. Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_scsi.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c index b70f71b5c1f7..a2cd22728b0f 100644 --- a/drivers/scsi/lpfc/lpfc_scsi.c +++ b/drivers/scsi/lpfc/lpfc_scsi.c @@ -3961,7 +3961,8 @@ lpfc_update_cmf_cmd(struct lpfc_hba *phba, uint32_t size) int cpu; /* At this point we are either LPFC_CFG_MANAGED or LPFC_CFG_MONITOR */ - if (phba->cmf_active_mode == LPFC_CFG_MANAGED) { + if (phba->cmf_active_mode == LPFC_CFG_MANAGED && + phba->cmf_max_bytes_per_interval) { total = 0; for_each_present_cpu(cpu) { cgs = per_cpu_ptr(phba->cmf_stat, cpu); From patchwork Fri Sep 10 23:31:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 509351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2094FC433F5 for ; Fri, 10 Sep 2021 23:32:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0B77C61206 for ; Fri, 10 Sep 2021 23:32:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234893AbhIJXdu (ORCPT ); Fri, 10 Sep 2021 19:33:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38616 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234908AbhIJXdf (ORCPT ); Fri, 10 Sep 2021 19:33:35 -0400 Received: from mail-pl1-x62f.google.com (mail-pl1-x62f.google.com [IPv6:2607:f8b0:4864:20::62f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8EE79C0613C1 for ; Fri, 10 Sep 2021 16:32:20 -0700 (PDT) Received: by mail-pl1-x62f.google.com with SMTP id n4so2119927plh.9 for ; Fri, 10 Sep 2021 16:32:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9hYHMJHZFH8ost3B+sxBQxA5bDUW1toQYgOsMCew8+Y=; b=Iw9J4Llqsf1SWzl4Tfjz++4NKSmO23J2WIsItbjuVGrL3REWp4tGIXDiVeeMrXhSua E7I1ScoOQJw6I/RhT7l8v1MNkeZDwzUgwjzSJyOwqjhctrh/7SRPA/uII7aQQF0eMQWw +YCRRS8U5S+5ZP+fdSAUJ0d6z3ugese0KVW/eejVGmC9BIOy/zz4MYtlKKr0VghWFqfD Eu9QESacZwu+bjvmWKoPfybVj+Fk5mIzfpY53pgoJDOy1aO2TI+JUJibba25lNzjXWaK bBwmNA0xXac2y/4xK04bqikWHLj74BjnxgZRT6svZS9DSU6v9bwyqY6G6O9ytUMNIZIu KP9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9hYHMJHZFH8ost3B+sxBQxA5bDUW1toQYgOsMCew8+Y=; b=Sf8ro/CVjTYOuU4l93VNln1Y1XkvVmzklgv4Bzjqtl1Op4uHhtLwp1SLe1S5c7Hnfr k+4J8nhNrdzjq2g/NVAed3MBdo5ppXD4bmacfCfD5S9pkbYBtt9vVJP+fhh0WlXupG7y M8YmVOn2i1oNb8mvXvWP8n/++LwMAFkUqokrreCOmJe3ScXMkqyxCgPCCNaQNnTyXmdX qjcPxRgN5m8AN17mnmFU90SBlTAFn32QLGWF+WHxQDp2sucxRwVI9bep2lOy/vEVEj4N dW5RHda6aLUFGA2SL3lCnT0ST0LIBtXu0vy+/J0TQGV1hb5H5UN0etRPWphssfQb7Cvf 6XhQ== X-Gm-Message-State: AOAM533E7GmKrYSPZe4W0uS9Gtm3gw/fyp/+saw4JRaV+1MkjU8NoW4X QVflBDkVi8lvcsHlzEySKGVb187cpqZpUTC1 X-Google-Smtp-Source: ABdhPJyVpuYacdUihedggnrH4DJWRc4HgmSWw2DI19r4EkBDdgqysDzyfeTmKcRML2Vax0c4G/o+CA== X-Received: by 2002:a17:90a:ae18:: with SMTP id t24mr94695pjq.92.1631316740040; Fri, 10 Sep 2021 16:32:20 -0700 (PDT) Received: from mail-ash-it-01.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id o15sm11325pfk.143.2021.09.10.16.32.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Sep 2021 16:32:19 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH 12/14] lpfc: Zero CGN stats only during initial driver load and stat reset Date: Fri, 10 Sep 2021 16:31:57 -0700 Message-Id: <20210910233159.115896-13-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210910233159.115896-1-jsmart2021@gmail.com> References: <20210910233159.115896-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Currently congestion management framework results are cleared whenever the framework settings changed (such as it being turned off then back on). This unfortunately means prior stats, rolled up to higher time windows lose meaning. Change such that stats are not cleared. Thus they pause and resume with prior values still being considered. Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_init.c | 2 -- drivers/scsi/lpfc/lpfc_sli.c | 6 ++++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c index 4637531f3b02..d0e64233d273 100644 --- a/drivers/scsi/lpfc/lpfc_init.c +++ b/drivers/scsi/lpfc/lpfc_init.c @@ -13394,8 +13394,6 @@ lpfc_init_congestion_buf(struct lpfc_hba *phba) atomic_set(&phba->cgn_sync_alarm_cnt, 0); atomic_set(&phba->cgn_sync_warn_cnt, 0); - atomic64_set(&phba->cgn_acqe_stat.alarm, 0); - atomic64_set(&phba->cgn_acqe_stat.warn, 0); atomic_set(&phba->cgn_driver_evt_cnt, 0); atomic_set(&phba->cgn_latency_evt_cnt, 0); atomic64_set(&phba->cgn_latency_evt, 0); diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index 651e6ee64e88..34cf2bfcce07 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -7764,8 +7764,6 @@ lpfc_mbx_cmpl_cgn_set_ftrs(struct lpfc_hba *phba, LPFC_MBOXQ_t *pmb) /* Zero out Congestion Signal ACQE counter */ phba->cgn_acqe_cnt = 0; - atomic64_set(&phba->cgn_acqe_stat.warn, 0); - atomic64_set(&phba->cgn_acqe_stat.alarm, 0); acqe = bf_get(lpfc_mbx_set_feature_CGN_acqe_freq, &pmb->u.mqe.un.set_feature); @@ -8017,6 +8015,10 @@ lpfc_cmf_setup(struct lpfc_hba *phba) /* initialize congestion buffer info */ lpfc_init_congestion_buf(phba); lpfc_init_congestion_stat(phba); + + /* Zero out Congestion Signal counters */ + atomic64_set(&phba->cgn_acqe_stat.alarm, 0); + atomic64_set(&phba->cgn_acqe_stat.warn, 0); } rc = lpfc_sli4_cgn_params_read(phba); From patchwork Fri Sep 10 23:31:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 509350 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A4ADC433EF for ; Fri, 10 Sep 2021 23:32:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 553BA61212 for ; Fri, 10 Sep 2021 23:32:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234905AbhIJXd7 (ORCPT ); Fri, 10 Sep 2021 19:33:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38620 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234912AbhIJXdf (ORCPT ); Fri, 10 Sep 2021 19:33:35 -0400 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AEE14C0613D8 for ; Fri, 10 Sep 2021 16:32:21 -0700 (PDT) Received: by mail-pf1-x433.google.com with SMTP id j6so2936912pfa.4 for ; Fri, 10 Sep 2021 16:32:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bHCwhC7mQKc33ENrUNkrzG8hfgkoJMDaQ5+v8gvc1dk=; b=izSGTPSgXXjN4AMtnj3MWHLRG5kGLCt2gwuaMyqOyDbEZMytE+mlENxQ4vZF9/FhET y/yjJuQ0HXe5V5uu8FBee/0qnzZrCFsi+Q/hSDI+1GPbW6H+QVoNdMDCH8yO6Isj5NyN 0awxBtgsvLYv0pWhLD4FL0YCc9QudxcTeSoSl0vf22EVJm+9j2z/cRWXvQXpUNdaj5lm yw/7mHgqRG2vHw2Q8xB7LzqcXFtLSPvaxQMN2kwfwr6qFHbpXlYfti3pJo+LtqNfvg7E QmtdgkX6QyJlxa9s+nIF6BYVnoSfcy3JTZ1EUEe3ArK7qU+QSnUeW+bokvkXYxSdq099 ptZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bHCwhC7mQKc33ENrUNkrzG8hfgkoJMDaQ5+v8gvc1dk=; b=Y9E2dd8LYzdYcYer9NnJ21iiK5pKa26tuCmq9vQzrrkwcviN4tzPE25NLrPnSMQYH/ 50F9evB/dl++BeYcp0Lq6WfZWbIeKxboEBpnW+W6FrfrXrEyWeEMNS12K2mwsI7+Huxp 0ril0nEHOBjnFEcom3x1H/bWmQTzc2I5EQDAaaZlJ7vFMKHonUFRH6lLeKB6gAdyshDq TmoNh99pcy6rtSfugU+K65xbI5ogY5trN1E3hMkOy5FQW/rpkkcom+Qx7QHxfPDq7rN6 596Zjq1FtknUQTh9V4wbb081haU5++GwxeJvt60D0k2xWZ6GvmylTIVnsSRTAQccjhdh y4JQ== X-Gm-Message-State: AOAM5322cfl9fCsWsY1h6QQGxX8Z/gPigf8YU1zHMa8oHAnUNTOo/n5f IUj/HYHbRC9EZlpIhRS2ivDkzQOzQPtdcoDW X-Google-Smtp-Source: ABdhPJygbKbCEEAZk3X0ntsMZQFZJPWPZtnUfj4Tav74KkRh7BWsT3YbV+LcX5HcnA/NFwelNkgxVw== X-Received: by 2002:a05:6a00:80f:b0:416:1ddf:3ed7 with SMTP id m15-20020a056a00080f00b004161ddf3ed7mr10408764pfk.79.1631316740730; Fri, 10 Sep 2021 16:32:20 -0700 (PDT) Received: from mail-ash-it-01.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id o15sm11325pfk.143.2021.09.10.16.32.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Sep 2021 16:32:20 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH 13/14] lpfc: Improve PBDE checks during SGL processing Date: Fri, 10 Sep 2021 16:31:58 -0700 Message-Id: <20210910233159.115896-14-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210910233159.115896-1-jsmart2021@gmail.com> References: <20210910233159.115896-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The PBDE feature, setting payload buffer address explicitly in the WQE so it doesn't have to be fetched from the SGL, only makes sense when there is a single buffer for the I/O. When there are multiple buffers it actually hurts performance as the SGL subsequently has to be fetched. Rework the SGL logic to only use PBDE when a single buffer. Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_nvme.c | 12 +++++----- drivers/scsi/lpfc/lpfc_nvmet.c | 44 ++++++++++++++++------------------ drivers/scsi/lpfc/lpfc_scsi.c | 26 ++++++++++---------- 3 files changed, 40 insertions(+), 42 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_nvme.c b/drivers/scsi/lpfc/lpfc_nvme.c index 33266e1b24ab..69d3758dd8dc 100644 --- a/drivers/scsi/lpfc/lpfc_nvme.c +++ b/drivers/scsi/lpfc/lpfc_nvme.c @@ -1299,7 +1299,6 @@ lpfc_nvme_prep_io_dma(struct lpfc_vport *vport, struct sli4_sge *first_data_sgl; struct ulp_bde64 *bde; dma_addr_t physaddr = 0; - uint32_t num_bde = 0; uint32_t dma_len = 0; uint32_t dma_offset = 0; int nseg, i, j; @@ -1353,7 +1352,7 @@ lpfc_nvme_prep_io_dma(struct lpfc_vport *vport, } sgl->word2 = 0; - if ((num_bde + 1) == nseg) { + if (nseg == 1) { bf_set(lpfc_sli4_sge_last, sgl, 1); bf_set(lpfc_sli4_sge_type, sgl, LPFC_SGE_TYPE_DATA); @@ -1422,8 +1421,9 @@ lpfc_nvme_prep_io_dma(struct lpfc_vport *vport, j++; } - if (phba->cfg_enable_pbde) { - /* Use PBDE support for first SGL only, offset == 0 */ + + /* PBDE support for first data SGE only */ + if (nseg == 1 && phba->cfg_enable_pbde) { /* Words 13-15 */ bde = (struct ulp_bde64 *) &wqe->words[13]; @@ -1434,11 +1434,11 @@ lpfc_nvme_prep_io_dma(struct lpfc_vport *vport, bde->tus.f.bdeFlags = BUFF_TYPE_BDE_64; bde->tus.w = cpu_to_le32(bde->tus.w); - /* Word 11 */ + /* Word 11 - set PBDE bit */ bf_set(wqe_pbde, &wqe->generic.wqe_com, 1); } else { memset(&wqe->words[13], 0, (sizeof(uint32_t) * 3)); - bf_set(wqe_pbde, &wqe->generic.wqe_com, 0); + /* Word 11 - PBDE bit disabled by default template */ } } else { diff --git a/drivers/scsi/lpfc/lpfc_nvmet.c b/drivers/scsi/lpfc/lpfc_nvmet.c index 6e3dd0b9bcfa..731802527b81 100644 --- a/drivers/scsi/lpfc/lpfc_nvmet.c +++ b/drivers/scsi/lpfc/lpfc_nvmet.c @@ -2708,7 +2708,7 @@ lpfc_nvmet_prep_fcp_wqe(struct lpfc_hba *phba, struct ulp_bde64 *bde; dma_addr_t physaddr; int i, cnt, nsegs; - int do_pbde; + bool use_pbde = false; int xc = 1; if (!lpfc_is_link_up(phba)) { @@ -2816,9 +2816,6 @@ lpfc_nvmet_prep_fcp_wqe(struct lpfc_hba *phba, if (!xc) bf_set(wqe_xc, &wqe->fcp_tsend.wqe_com, 0); - /* Word 11 - set sup, irsp, irsplen later */ - do_pbde = 0; - /* Word 12 */ wqe->fcp_tsend.fcp_data_len = rsp->transfer_length; @@ -2896,12 +2893,13 @@ lpfc_nvmet_prep_fcp_wqe(struct lpfc_hba *phba, if (!xc) bf_set(wqe_xc, &wqe->fcp_treceive.wqe_com, 0); - /* Word 11 - set pbde later */ - if (phba->cfg_enable_pbde) { - do_pbde = 1; + /* Word 11 - check for pbde */ + if (nsegs == 1 && phba->cfg_enable_pbde) { + use_pbde = true; + /* Word 11 - PBDE bit already preset by template */ } else { + /* Overwrite default template setting */ bf_set(wqe_pbde, &wqe->fcp_treceive.wqe_com, 0); - do_pbde = 0; } /* Word 12 */ @@ -2972,7 +2970,6 @@ lpfc_nvmet_prep_fcp_wqe(struct lpfc_hba *phba, ((rsp->rsplen >> 2) - 1)); memcpy(&wqe->words[16], rsp->rspaddr, rsp->rsplen); } - do_pbde = 0; /* Word 12 */ wqe->fcp_trsp.rsvd_12_15[0] = 0; @@ -3007,23 +3004,24 @@ lpfc_nvmet_prep_fcp_wqe(struct lpfc_hba *phba, bf_set(lpfc_sli4_sge_last, sgl, 1); sgl->word2 = cpu_to_le32(sgl->word2); sgl->sge_len = cpu_to_le32(cnt); - if (i == 0) { - bde = (struct ulp_bde64 *)&wqe->words[13]; - if (do_pbde) { - /* Words 13-15 (PBDE) */ - bde->addrLow = sgl->addr_lo; - bde->addrHigh = sgl->addr_hi; - bde->tus.f.bdeSize = - le32_to_cpu(sgl->sge_len); - bde->tus.f.bdeFlags = BUFF_TYPE_BDE_64; - bde->tus.w = cpu_to_le32(bde->tus.w); - } else { - memset(bde, 0, sizeof(struct ulp_bde64)); - } - } sgl++; ctxp->offset += cnt; } + + bde = (struct ulp_bde64 *)&wqe->words[13]; + if (use_pbde) { + /* decrement sgl ptr backwards once to first data sge */ + sgl--; + + /* Words 13-15 (PBDE) */ + bde->addrLow = sgl->addr_lo; + bde->addrHigh = sgl->addr_hi; + bde->tus.f.bdeSize = le32_to_cpu(sgl->sge_len); + bde->tus.f.bdeFlags = BUFF_TYPE_BDE_64; + bde->tus.w = cpu_to_le32(bde->tus.w); + } else { + memset(bde, 0, sizeof(struct ulp_bde64)); + } ctxp->state = LPFC_NVME_STE_DATA; ctxp->entry_cnt++; return nvmewqe; diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c index a2cd22728b0f..078fbea3f436 100644 --- a/drivers/scsi/lpfc/lpfc_scsi.c +++ b/drivers/scsi/lpfc/lpfc_scsi.c @@ -3235,7 +3235,6 @@ lpfc_scsi_prep_dma_buf_s4(struct lpfc_hba *phba, struct lpfc_io_buf *lpfc_cmd) struct lpfc_vport *vport = phba->pport; union lpfc_wqe128 *wqe = &pwqeq->wqe; dma_addr_t physaddr; - uint32_t num_bde = 0; uint32_t dma_len; uint32_t dma_offset = 0; int nseg, i, j; @@ -3297,7 +3296,7 @@ lpfc_scsi_prep_dma_buf_s4(struct lpfc_hba *phba, struct lpfc_io_buf *lpfc_cmd) j = 2; for (i = 0; i < nseg; i++) { sgl->word2 = 0; - if ((num_bde + 1) == nseg) { + if (nseg == 1) { bf_set(lpfc_sli4_sge_last, sgl, 1); bf_set(lpfc_sli4_sge_type, sgl, LPFC_SGE_TYPE_DATA); @@ -3366,13 +3365,15 @@ lpfc_scsi_prep_dma_buf_s4(struct lpfc_hba *phba, struct lpfc_io_buf *lpfc_cmd) j++; } - /* - * Setup the first Payload BDE. For FCoE we just key off - * Performance Hints, for FC we use lpfc_enable_pbde. - * We populate words 13-15 of IOCB/WQE. + + /* PBDE support for first data SGE only. + * For FCoE, we key off Performance Hints. + * For FC, we key off lpfc_enable_pbde. */ - if ((phba->sli3_options & LPFC_SLI4_PERFH_ENABLED) || - phba->cfg_enable_pbde) { + if (nseg == 1 && + ((phba->sli3_options & LPFC_SLI4_PERFH_ENABLED) || + phba->cfg_enable_pbde)) { + /* Words 13-15 */ bde = (struct ulp_bde64 *) &wqe->words[13]; bde->addrLow = first_data_sgl->addr_lo; @@ -3382,12 +3383,15 @@ lpfc_scsi_prep_dma_buf_s4(struct lpfc_hba *phba, struct lpfc_io_buf *lpfc_cmd) bde->tus.f.bdeFlags = BUFF_TYPE_BDE_64; bde->tus.w = cpu_to_le32(bde->tus.w); + /* Word 11 - set PBDE bit */ + bf_set(wqe_pbde, &wqe->generic.wqe_com, 1); } else { memset(&wqe->words[13], 0, (sizeof(uint32_t) * 3)); + /* Word 11 - PBDE bit disabled by default template */ } } else { sgl += 1; - /* clear the last flag in the fcp_rsp map entry */ + /* set the last flag in the fcp_rsp map entry */ sgl->word2 = le32_to_cpu(sgl->word2); bf_set(lpfc_sli4_sge_last, sgl, 1); sgl->word2 = cpu_to_le32(sgl->word2); @@ -3400,10 +3404,6 @@ lpfc_scsi_prep_dma_buf_s4(struct lpfc_hba *phba, struct lpfc_io_buf *lpfc_cmd) } } - /* Word 11 */ - if (phba->cfg_enable_pbde) - bf_set(wqe_pbde, &wqe->generic.wqe_com, 1); - /* * Finish initializing those IOCB fields that are dependent on the * scsi_cmnd request_buffer. Note that for SLI-2 the bdeSize is From patchwork Fri Sep 10 23:31:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 509001 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6825CC433F5 for ; Fri, 10 Sep 2021 23:33:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3DD5A61212 for ; Fri, 10 Sep 2021 23:33:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234898AbhIJXeC (ORCPT ); Fri, 10 Sep 2021 19:34:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38622 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234913AbhIJXdf (ORCPT ); Fri, 10 Sep 2021 19:33:35 -0400 Received: from mail-pg1-x530.google.com (mail-pg1-x530.google.com [IPv6:2607:f8b0:4864:20::530]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EFD7CC0613D9 for ; Fri, 10 Sep 2021 16:32:21 -0700 (PDT) Received: by mail-pg1-x530.google.com with SMTP id q68so3218511pga.9 for ; Fri, 10 Sep 2021 16:32:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=V3CtqB54aGUlAB2YSezJgPFCNJSusZEaGV+8iCSjgiM=; b=hnK7vvCW8aUzBgtksP64l6d1xKM36TUuXJjsgV6BmRey2mLKkXSbqlPOUJEj/U4QnE p03RnWZHYuvuXkjI6ffpg/0PyOJkwD8duELdirqC1f5sVSYi3PQb/jvw71r4sB0/qtNA oVBZziBQD3EImoJeaCi60nMwGr5bvAgSf2tjizot101crBn2cVnSU4Mr463b9e2pMF9I nd3mzK7/kPbgUnAZAT186AG+Czr7mL+sZ7hNGBpTc/JFPRytUMglmLL9EMwbiT20KGtd 4AJY1PMuBAh8pvlCNfhgsk/kACgsL+0Sz2oUE7GBESFFBdJOzm2o+P6GcGL9lPzB4OyV jXPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=V3CtqB54aGUlAB2YSezJgPFCNJSusZEaGV+8iCSjgiM=; b=ltkdv8cqZbnl+ksg7NU3NmeBP30Qr3AIaYSO68MyaMMJT8CEsBH9IAMWYzMQgSax7d dmz4vyVwIGKShZ1isDQmInL3PvRXa6HLWkcWIvdCeo51H/J3JSVDDhkWNG48PPJKg5bc pDUCwRjXbBqV9uHNs/EpdGJUIwxwn44kQfzOwkb6zMGOjicfb4T3TBZmHYaWqep8KlHi Y+5PSIEJOvURYfsz2KgOxpPtxJWFfTmCtpqz1uznVu9C8qLzMJcvCdkU8vBt1EbSUsSW /FZHWM+bpqQAfzNUWoHwqa984SnhhD1QPVl9BOgFv9Z0WoHaWD4ewuchyHwYiVwCmXmv +jrA== X-Gm-Message-State: AOAM5339UhAguV0xVObhULYf+ycHL6PeOgND6drTAbQin8bSPlx+oSKJ SS1lL1nBRFwH2caCUlTHCxT1aTT788CCDKLR X-Google-Smtp-Source: ABdhPJz8RG5K52o3OgojPj8xkuiBUvp2HqGKgLIeDvjiZEFDPzfLN+bnOEGqLOR/aS1bqwIhT1mLDQ== X-Received: by 2002:a05:6a00:1823:b0:42a:ee71:d74a with SMTP id y35-20020a056a00182300b0042aee71d74amr1599pfa.63.1631316741412; Fri, 10 Sep 2021 16:32:21 -0700 (PDT) Received: from mail-ash-it-01.broadcom.com ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id o15sm11325pfk.143.2021.09.10.16.32.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 10 Sep 2021 16:32:21 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH 14/14] lpfc: Update lpfc version to 14.0.0.2 Date: Fri, 10 Sep 2021 16:31:59 -0700 Message-Id: <20210910233159.115896-15-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210910233159.115896-1-jsmart2021@gmail.com> References: <20210910233159.115896-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Update lpfc version to 14.0.0.2 Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_version.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/scsi/lpfc/lpfc_version.h b/drivers/scsi/lpfc/lpfc_version.h index a7aba7833425..dec71775f677 100644 --- a/drivers/scsi/lpfc/lpfc_version.h +++ b/drivers/scsi/lpfc/lpfc_version.h @@ -20,7 +20,7 @@ * included with this package. * *******************************************************************/ -#define LPFC_DRIVER_VERSION "14.0.0.1" +#define LPFC_DRIVER_VERSION "14.0.0.2" #define LPFC_DRIVER_NAME "lpfc" /* Used for SLI 2/3 */