From patchwork Fri Aug 13 23:00:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 496861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8E7CC432BE for ; Fri, 13 Aug 2021 23:00:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A098760F51 for ; Fri, 13 Aug 2021 23:00:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235359AbhHMXBV (ORCPT ); Fri, 13 Aug 2021 19:01:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41510 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235342AbhHMXBV (ORCPT ); Fri, 13 Aug 2021 19:01:21 -0400 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5154C061756 for ; Fri, 13 Aug 2021 16:00:53 -0700 (PDT) Received: by mail-pj1-x102d.google.com with SMTP id m24-20020a17090a7f98b0290178b1a81700so18109185pjl.4 for ; Fri, 13 Aug 2021 16:00:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RIVK96bKPkdnIkHGr5K4QFuikQwUgsi8aLcsQ7LYOlc=; b=Z8kw2Q9JfaBM6rzGOVE3+qusTTo0vei0EtBoOjaF6qGW2URWq6oysZIGcWHz7fyLYo EgIBfCKTMjFD0aj1vXVakRBw3vXy5FEiFxr3hl0ScDTsIFUpZaG3wLO1ksWSIQwqhZAH KPDE3225j/sVoG/LteIKIHMs0HQVSz9beOD13uKWTxX7AT66qp3kWpIFm9ZXkecwwYyM v5nwNziG1Ervq+oIxWp2OMQEe3TuManB16g2qH7ELKycP2JHYWoBJUI2AGQSq17LMT3n t4Owh5u0TJcqXgMdtFuDq9qqtC4P5QnHT2TAbw8tbZ3UvkxHNDfaO4Gfqz7LZl5/2V7X RBDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RIVK96bKPkdnIkHGr5K4QFuikQwUgsi8aLcsQ7LYOlc=; b=ildcuGyOEuUF6NLzIVRg7yzUhseb3fEjQ92vZ7hwVBsG6M5fQ+uccaTGucR+iqymyF 1MCix+FG7xCQ0KeBO0wtpxSB5bR15obtGIP819i9KinFUDdtQft7LxqAmoTer+Iv7I4v Z0MFa+1Zy0fgs6uvTy3rhZwjnnai2g6RUoX6aAMt1lgDDrxw8k+i31ATkrQii7ViHlO7 pK0FqcL7oEUET0HIxp4a3PZku7fvukzW2fRxBm8//QGt5/+hz2ewEC8bnd5cw715Y0h1 b9Kub8zPZyeRJ6ny/QWFfyIduCrLWJYg/NB3FiaKaJXx8j8lLypDTTHDg4xBriMk+J1c XsPQ== X-Gm-Message-State: AOAM530kz0ewN937/WOqV2LSoVI9uYNhslBm9uu5n80Qt1sjCYpeNDpH gF8Kpan+Z52oi5qyExJ7Vv/RDnIg60s= X-Google-Smtp-Source: ABdhPJzbhYmObsUSRVO6O1R/VqKQkjQTMBzSoo/7NAnMjTu7Ik9NSahPe2kTTTlBjygTVeu5ZWjr2w== X-Received: by 2002:a62:6042:0:b029:3cd:d392:643 with SMTP id u63-20020a6260420000b02903cdd3920643mr4587081pfb.43.1628895653200; Fri, 13 Aug 2021 16:00:53 -0700 (PDT) Received: from localhost.localdomain.localdomain ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id e8sm4001997pgg.31.2021.08.13.16.00.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 16:00:53 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH v2 02/16] lpfc: Add SET_HOST_DATA mbox cmd to pass date/time info to firmware Date: Fri, 13 Aug 2021 16:00:25 -0700 Message-Id: <20210813230039.110546-3-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210813230039.110546-1-jsmart2021@gmail.com> References: <20210813230039.110546-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Implement the SET_HOST_DATA mbox command to set date / time during initialization. It is used by the firmware for various purposes including congestion management and firmware dumps. Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_hw4.h | 30 ++++++++++++++++++++- drivers/scsi/lpfc/lpfc_sli.c | 51 +++++++++++++++++++++++++++++++++++- 2 files changed, 79 insertions(+), 2 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_hw4.h b/drivers/scsi/lpfc/lpfc_hw4.h index aadbb0de629d..658b9c558237 100644 --- a/drivers/scsi/lpfc/lpfc_hw4.h +++ b/drivers/scsi/lpfc/lpfc_hw4.h @@ -3427,12 +3427,40 @@ struct lpfc_mbx_set_feature { #define LPFC_SET_HOST_OS_DRIVER_VERSION 0x2 +#define LPFC_SET_HOST_DATE_TIME 0x4 + +struct lpfc_mbx_set_host_date_time { + uint32_t word6; +#define lpfc_mbx_set_host_month_WORD word6 +#define lpfc_mbx_set_host_month_SHIFT 16 +#define lpfc_mbx_set_host_month_MASK 0xFF +#define lpfc_mbx_set_host_day_WORD word6 +#define lpfc_mbx_set_host_day_SHIFT 8 +#define lpfc_mbx_set_host_day_MASK 0xFF +#define lpfc_mbx_set_host_year_WORD word6 +#define lpfc_mbx_set_host_year_SHIFT 0 +#define lpfc_mbx_set_host_year_MASK 0xFF + uint32_t word7; +#define lpfc_mbx_set_host_hour_WORD word7 +#define lpfc_mbx_set_host_hour_SHIFT 16 +#define lpfc_mbx_set_host_hour_MASK 0xFF +#define lpfc_mbx_set_host_min_WORD word7 +#define lpfc_mbx_set_host_min_SHIFT 8 +#define lpfc_mbx_set_host_min_MASK 0xFF +#define lpfc_mbx_set_host_sec_WORD word7 +#define lpfc_mbx_set_host_sec_SHIFT 0 +#define lpfc_mbx_set_host_sec_MASK 0xFF +}; + struct lpfc_mbx_set_host_data { #define LPFC_HOST_OS_DRIVER_VERSION_SIZE 48 struct mbox_header header; uint32_t param_id; uint32_t param_len; - uint8_t data[LPFC_HOST_OS_DRIVER_VERSION_SIZE]; + union { + uint8_t data[LPFC_HOST_OS_DRIVER_VERSION_SIZE]; + struct lpfc_mbx_set_host_date_time tm; + } un; }; struct lpfc_mbx_set_trunk_mode { diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index 47dd13719901..9ff4abb966af 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -7369,7 +7369,7 @@ lpfc_set_host_data(struct lpfc_hba *phba, LPFC_MBOXQ_t *mbox) mbox->u.mqe.un.set_host_data.param_id = LPFC_SET_HOST_OS_DRIVER_VERSION; mbox->u.mqe.un.set_host_data.param_len = LPFC_HOST_OS_DRIVER_VERSION_SIZE; - snprintf(mbox->u.mqe.un.set_host_data.data, + snprintf(mbox->u.mqe.un.set_host_data.un.data, LPFC_HOST_OS_DRIVER_VERSION_SIZE, "Linux %s v"LPFC_DRIVER_VERSION, (phba->hba_flag & HBA_FCOE_MODE) ? "FCoE" : "FC"); @@ -7499,6 +7499,51 @@ static void lpfc_sli4_dip(struct lpfc_hba *phba) } } +static int +lpfc_set_host_tm(struct lpfc_hba *phba) +{ + LPFC_MBOXQ_t *mboxq; + uint32_t len, rc; + struct timespec64 cur_time; + struct tm broken; + uint32_t month, day, year; + uint32_t hour, minute, second; + struct lpfc_mbx_set_host_date_time *tm; + + mboxq = (LPFC_MBOXQ_t *)mempool_alloc(phba->mbox_mem_pool, GFP_KERNEL); + if (!mboxq) + return -ENOMEM; + + len = sizeof(struct lpfc_mbx_set_host_data) - + sizeof(struct lpfc_sli4_cfg_mhdr); + lpfc_sli4_config(phba, mboxq, LPFC_MBOX_SUBSYSTEM_COMMON, + LPFC_MBOX_OPCODE_SET_HOST_DATA, len, + LPFC_SLI4_MBX_EMBED); + + mboxq->u.mqe.un.set_host_data.param_id = LPFC_SET_HOST_DATE_TIME; + mboxq->u.mqe.un.set_host_data.param_len = + sizeof(struct lpfc_mbx_set_host_date_time); + tm = &mboxq->u.mqe.un.set_host_data.un.tm; + ktime_get_real_ts64(&cur_time); + time64_to_tm(cur_time.tv_sec, 0, &broken); + month = broken.tm_mon + 1; + day = broken.tm_mday; + year = broken.tm_year - 100; + hour = broken.tm_hour; + minute = broken.tm_min; + second = broken.tm_sec; + bf_set(lpfc_mbx_set_host_month, tm, month); + bf_set(lpfc_mbx_set_host_day, tm, day); + bf_set(lpfc_mbx_set_host_year, tm, year); + bf_set(lpfc_mbx_set_host_hour, tm, hour); + bf_set(lpfc_mbx_set_host_min, tm, minute); + bf_set(lpfc_mbx_set_host_sec, tm, second); + + rc = lpfc_sli_issue_mbox(phba, mboxq, MBX_POLL); + mempool_free(mboxq, phba->mbox_mem_pool); + return rc; +} + /** * lpfc_sli4_hba_setup - SLI4 device initialization PCI function * @phba: Pointer to HBA context object. @@ -7588,6 +7633,10 @@ lpfc_sli4_hba_setup(struct lpfc_hba *phba) goto out_free_mbox; } + rc = lpfc_set_host_tm(phba); + lpfc_printf_log(phba, KERN_ERR, LOG_MBOX | LOG_INIT, + "6468 Set host date / time: Status x%x:\n", rc); + /* * Continue initialization with default values even if driver failed * to read FCoE param config regions, only read parameters if the From patchwork Fri Aug 13 23:00:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 496860 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06125C432BE for ; Fri, 13 Aug 2021 23:01:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DBB4160FC4 for ; Fri, 13 Aug 2021 23:00:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235411AbhHMXB0 (ORCPT ); Fri, 13 Aug 2021 19:01:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235390AbhHMXBZ (ORCPT ); Fri, 13 Aug 2021 19:01:25 -0400 Received: from mail-pj1-x1035.google.com (mail-pj1-x1035.google.com [IPv6:2607:f8b0:4864:20::1035]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B6F1FC061756 for ; Fri, 13 Aug 2021 16:00:58 -0700 (PDT) Received: by mail-pj1-x1035.google.com with SMTP id hv22-20020a17090ae416b0290178c579e424so18126017pjb.3 for ; Fri, 13 Aug 2021 16:00:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KGKNdJ89ylg0AP+9Ko+8q/ExG+56dWz3dDRW6iBXN2M=; b=NmQ6W4ioD9ZK3pmVAO+u6ZP4EWIx/igqRubvc55HHtmQ1cco9xfv52KGQf1n2a1K1h F5RILCybq1mhCiMqdTonoNOWw/BWjhJna8ClJSGzCbSngkORsq46Xh0GTZ32koaw3vj3 7sHVFR4tuhpq1iQxmutiFG/G0PHcO91hmb9UoK1Blrl5BtGaRZOzRKvYSlJhDb9J8uIy Z74tmzz+mlsmcuTA6VLOvr0QRJ1SIS66Cz0UQtGGF19/ovKq8dD5/FLu8OxdtNA+S+Cd jwUCMgjWRjMukkxexa5wAAeX0Uu+92XylshulNcjN5eGRP/pWk26Cbn0/kxsV32MSaFX WXDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KGKNdJ89ylg0AP+9Ko+8q/ExG+56dWz3dDRW6iBXN2M=; b=inleT7rj5fcgRF7LRTOAw1hzCNzpFFrUkb+3w5gL1smMkhVSytzdOvzqUAImcDWhMt MVOZvnQNH9JBc9PXvvJjg807gmHWJJJRukoPObYWfcX3pxnalgD/G/ispklkJw7c88Ey O9BpYop2EtfATrskKNBrQ5njsfTmCg7/WYYdbpxAQcRqVUdf8GegbnNKSg7JGy76r9HO qhYUkxflqMGo9dKy2DsxXaSVVteMyznDaRpokMXgyOf2zxbOu3jWyjM7AvwWy5QNTv2+ 4tHHMC0YM3QearFDL44ihz0cuz9ZOGsa5q2IsgCgFhVrNwTE96iapO8PnxjrsYHQohyh nu6A== X-Gm-Message-State: AOAM532c7/yjx6fLSEUp1+bN/+jWEny0/fF4fXVHYH7bCI+/8BKNwm4v idne5ZMkeKXqOa65gTFccbzm4BHrHck= X-Google-Smtp-Source: ABdhPJynpOnOgpSNcXLJCzVEUXv/DkCdeYpa+A2RZDzTkRY0jEDOYWN59v8RREaWcueih51eTIYr+w== X-Received: by 2002:a05:6a00:84e:b029:3ae:5c9:a48d with SMTP id q14-20020a056a00084eb02903ae05c9a48dmr4678012pfk.20.1628895658069; Fri, 13 Aug 2021 16:00:58 -0700 (PDT) Received: from localhost.localdomain.localdomain ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id e8sm4001997pgg.31.2021.08.13.16.00.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 16:00:57 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH v2 04/16] lpfc: Expand FPIN and RDF receive logging Date: Fri, 13 Aug 2021 16:00:27 -0700 Message-Id: <20210813230039.110546-5-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210813230039.110546-1-jsmart2021@gmail.com> References: <20210813230039.110546-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Expand FPIN logging: - Display Attached Port Names for Link Integrity and Peer Congestion events - Log Delivery, Peer Congestion, and Congestion events - Sanity check FPIN descriptor lengths when processing FPIN descriptors. Log RDF events when congestion logging is enabled Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_els.c | 349 +++++++++++++++++++++++++++++++---- drivers/scsi/lpfc/lpfc_hw4.h | 14 ++ 2 files changed, 322 insertions(+), 41 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c index 08ae2b12b92c..097f132fc35b 100644 --- a/drivers/scsi/lpfc/lpfc_els.c +++ b/drivers/scsi/lpfc/lpfc_els.c @@ -3260,7 +3260,7 @@ lpfc_cmpl_els_disc_cmd(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, irsp->ulpStatus, irsp->un.ulpWord[4], irsp->un.elsreq64.remoteID); /* ELS cmd tag completes */ - lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS, + lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS | LOG_CGN_MGMT, "0217 ELS cmd tag x%x completes Data: x%x x%x x%x " "x%x\n", irsp->ulpIoTag, irsp->ulpStatus, @@ -3319,11 +3319,12 @@ lpfc_cmpl_els_disc_cmd(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, for (i = 0; i < ELS_RDF_REG_TAG_CNT && i < be32_to_cpu(prdf->reg_d1.reg_desc.count); i++) - lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS, - "4677 Fabric RDF Notification Grant Data: " - "0x%08x\n", - be32_to_cpu( - prdf->reg_d1.desc_tags[i])); + lpfc_printf_vlog(vport, KERN_INFO, + LOG_ELS | LOG_CGN_MGMT, + "4677 Fabric RDF Notification Grant " + "Data: 0x%08x\n", + be32_to_cpu( + prdf->reg_d1.desc_tags[i])); } out: @@ -3689,7 +3690,7 @@ lpfc_issue_els_rdf(struct lpfc_vport *vport, uint8_t retry) prdf->reg_d1.desc_tags[2] = cpu_to_be32(ELS_DTAG_PEER_CONGEST); prdf->reg_d1.desc_tags[3] = cpu_to_be32(ELS_DTAG_CONGESTION); - lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS, + lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS | LOG_CGN_MGMT, "6444 Xmit RDF to remote NPORT x%x\n", ndlp->nlp_DID); @@ -3733,7 +3734,7 @@ lpfc_els_rcv_rdf(struct lpfc_vport *vport, struct lpfc_iocbq *cmdiocb, { /* Send LS_ACC */ if (lpfc_els_rsp_acc(vport, ELS_CMD_RDF, cmdiocb, ndlp, NULL)) { - lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS, + lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS | LOG_CGN_MGMT, "1623 Failed to RDF_ACC from x%x for x%x\n", ndlp->nlp_DID, vport->fc_myDID); return -EIO; @@ -3741,7 +3742,7 @@ lpfc_els_rcv_rdf(struct lpfc_vport *vport, struct lpfc_iocbq *cmdiocb, /* Issue new RDF for reregistering */ if (lpfc_issue_els_rdf(vport, 0)) { - lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS, + lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS | LOG_CGN_MGMT, "2623 Failed to re register RDF for x%x\n", vport->fc_myDID); return -EIO; @@ -8693,44 +8694,254 @@ DECLARE_ENUM2STR_LOOKUP(lpfc_get_tlv_dtag_nm, fc_ls_tlv_dtag, DECLARE_ENUM2STR_LOOKUP(lpfc_get_fpin_li_event_nm, fc_fpin_li_event_types, FC_FPIN_LI_EVT_TYPES_INIT); +DECLARE_ENUM2STR_LOOKUP(lpfc_get_fpin_deli_event_nm, fc_fpin_deli_event_types, + FC_FPIN_DELI_EVT_TYPES_INIT); + +DECLARE_ENUM2STR_LOOKUP(lpfc_get_fpin_congn_event_nm, fc_fpin_congn_event_types, + FC_FPIN_CONGN_EVT_TYPES_INIT); + +DECLARE_ENUM2STR_LOOKUP(lpfc_get_fpin_congn_severity_nm, + fc_fpin_congn_severity_types, + FC_FPIN_CONGN_SEVERITY_INIT); + + +/** + * lpfc_display_fpin_wwpn - Display WWPNs accessible by the attached port + * @phba: Pointer to phba object. + * @wwnlist: Pointer to list of WWPNs in FPIN payload + * @cnt: count of WWPNs in FPIN payload + * + * This routine is called by LI and PC descriptors. + * Limit the number of WWPNs displayed to 6 log messages, 6 per log message + */ +static void +lpfc_display_fpin_wwpn(struct lpfc_hba *phba, __be64 *wwnlist, u32 cnt) +{ + char buf[LPFC_FPIN_WWPN_LINE_SZ]; + __be64 wwn; + u64 wwpn; + int i, len; + int line = 0; + int wcnt = 0; + bool endit = false; + + len = scnprintf(buf, LPFC_FPIN_WWPN_LINE_SZ, "Accessible WWPNs:"); + for (i = 0; i < cnt; i++) { + /* Are we on the last WWPN */ + if (i == (cnt - 1)) + endit = true; + + /* Extract the next WWPN from the payload */ + wwn = *wwnlist++; + wwpn = be64_to_cpu(wwn); + len += scnprintf(buf + len, LPFC_FPIN_WWPN_LINE_SZ, + " %016llx", wwpn); + + /* Log a message if we are on the last WWPN + * or if we hit the max allowed per message. + */ + wcnt++; + if (wcnt == LPFC_FPIN_WWPN_LINE_CNT || endit) { + buf[len] = 0; + lpfc_printf_log(phba, KERN_INFO, LOG_ELS, + "4686 %s\n", buf); + + /* Check if we reached the last WWPN */ + if (endit) + return; + + /* Limit the number of log message displayed per FPIN */ + line++; + if (line == LPFC_FPIN_WWPN_NUM_LINE) { + lpfc_printf_log(phba, KERN_INFO, LOG_ELS, + "4687 %d WWPNs Truncated\n", + cnt - i - 1); + return; + } + + /* Start over with next log message */ + wcnt = 0; + len = scnprintf(buf, LPFC_FPIN_WWPN_LINE_SZ, + "Additional WWPNs:"); + } + } +} + /** * lpfc_els_rcv_fpin_li - Process an FPIN Link Integrity Event. - * @vport: Pointer to vport object. + * @phba: Pointer to phba object. * @tlv: Pointer to the Link Integrity Notification Descriptor. * - * This function processes a link integrity FPIN event by - * logging a message + * This function processes a Link Integrity FPIN event by logging a message. **/ static void -lpfc_els_rcv_fpin_li(struct lpfc_vport *vport, struct fc_tlv_desc *tlv) +lpfc_els_rcv_fpin_li(struct lpfc_hba *phba, struct fc_tlv_desc *tlv) { struct fc_fn_li_desc *li = (struct fc_fn_li_desc *)tlv; const char *li_evt_str; - u32 li_evt; + u32 li_evt, cnt; li_evt = be16_to_cpu(li->event_type); li_evt_str = lpfc_get_fpin_li_event_nm(li_evt); + cnt = be32_to_cpu(li->pname_count); - lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS, - "4680 FPIN Link Integrity %s (x%x) " - "Detecting PN x%016llx Attached PN x%016llx " - "Duration %d mSecs Count %d Port Cnt %d\n", - li_evt_str, li_evt, - be64_to_cpu(li->detecting_wwpn), - be64_to_cpu(li->attached_wwpn), - be32_to_cpu(li->event_threshold), - be32_to_cpu(li->event_count), - be32_to_cpu(li->pname_count)); + lpfc_printf_log(phba, KERN_INFO, LOG_ELS, + "4680 FPIN Link Integrity %s (x%x) " + "Detecting PN x%016llx Attached PN x%016llx " + "Duration %d mSecs Count %d Port Cnt %d\n", + li_evt_str, li_evt, + be64_to_cpu(li->detecting_wwpn), + be64_to_cpu(li->attached_wwpn), + be32_to_cpu(li->event_threshold), + be32_to_cpu(li->event_count), cnt); + + lpfc_display_fpin_wwpn(phba, (__be64 *)&li->pname_list, cnt); +} + +/** + * lpfc_els_rcv_fpin_del - Process an FPIN Delivery Event. + * @phba: Pointer to hba object. + * @tlv: Pointer to the Delivery Notification Descriptor TLV + * + * This function processes a Delivery FPIN event by logging a message. + **/ +static void +lpfc_els_rcv_fpin_del(struct lpfc_hba *phba, struct fc_tlv_desc *tlv) +{ + struct fc_fn_deli_desc *del = (struct fc_fn_deli_desc *)tlv; + const char *del_rsn_str; + u32 del_rsn; + __be32 *frame; + + del_rsn = be16_to_cpu(del->deli_reason_code); + del_rsn_str = lpfc_get_fpin_deli_event_nm(del_rsn); + + /* Skip over desc_tag/desc_len header to payload */ + frame = (__be32 *)(del + 1); + + lpfc_printf_log(phba, KERN_INFO, LOG_ELS, + "4681 FPIN Delivery %s (x%x) " + "Detecting PN x%016llx Attached PN x%016llx " + "DiscHdr0 x%08x " + "DiscHdr1 x%08x DiscHdr2 x%08x DiscHdr3 x%08x " + "DiscHdr4 x%08x DiscHdr5 x%08x\n", + del_rsn_str, del_rsn, + be64_to_cpu(del->detecting_wwpn), + be64_to_cpu(del->attached_wwpn), + be32_to_cpu(frame[0]), + be32_to_cpu(frame[1]), + be32_to_cpu(frame[2]), + be32_to_cpu(frame[3]), + be32_to_cpu(frame[4]), + be32_to_cpu(frame[5])); } +/** + * lpfc_els_rcv_fpin_peer_cgn - Process a FPIN Peer Congestion Event. + * @phba: Pointer to hba object. + * @tlv: Pointer to the Peer Congestion Notification Descriptor TLV + * + * This function processes a Peer Congestion FPIN event by logging a message. + **/ static void -lpfc_els_rcv_fpin(struct lpfc_vport *vport, struct fc_els_fpin *fpin, - u32 fpin_length) +lpfc_els_rcv_fpin_peer_cgn(struct lpfc_hba *phba, struct fc_tlv_desc *tlv) { - struct fc_tlv_desc *tlv; + struct fc_fn_peer_congn_desc *pc = (struct fc_fn_peer_congn_desc *)tlv; + const char *pc_evt_str; + u32 pc_evt, cnt; + + pc_evt = be16_to_cpu(pc->event_type); + pc_evt_str = lpfc_get_fpin_congn_event_nm(pc_evt); + cnt = be32_to_cpu(pc->pname_count); + + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT | LOG_ELS, + "4684 FPIN Peer Congestion %s (x%x) " + "Duration %d mSecs " + "Detecting PN x%016llx Attached PN x%016llx " + "Impacted Port Cnt %d\n", + pc_evt_str, pc_evt, + be32_to_cpu(pc->event_period), + be64_to_cpu(pc->detecting_wwpn), + be64_to_cpu(pc->attached_wwpn), + cnt); + + lpfc_display_fpin_wwpn(phba, (__be64 *)&pc->pname_list, cnt); +} + +/** + * lpfc_els_rcv_fpin_cgn - Process an FPIN Congestion notification + * @phba: Pointer to hba object. + * @tlv: Pointer to the Congestion Notification Descriptor TLV + * + * This function processes an FPIN Congestion Notifiction. The notification + * could be an Alarm or Warning. This routine feeds that data into driver's + * running congestion algorithm. It also processes the FPIN by + * logging a message. It returns 1 to indicate deliver this message + * to the upper layer or 0 to indicate don't deliver it. + **/ +static int +lpfc_els_rcv_fpin_cgn(struct lpfc_hba *phba, struct fc_tlv_desc *tlv) +{ + struct fc_fn_congn_desc *cgn = (struct fc_fn_congn_desc *)tlv; + const char *cgn_evt_str; + u32 cgn_evt; + const char *cgn_sev_str; + u32 cgn_sev; + bool nm_log = false; + int rc = 1; + + cgn_evt = be16_to_cpu(cgn->event_type); + cgn_evt_str = lpfc_get_fpin_congn_event_nm(cgn_evt); + cgn_sev = cgn->severity; + cgn_sev_str = lpfc_get_fpin_congn_severity_nm(cgn_sev); + + /* The driver only takes action on a Credit Stall or Oversubscription + * event type to engage the IO algorithm. The driver prints an + * unmaskable message only for Lost Credit and Credit Stall. + * TODO: Still need to have definition of host action on clear, + * lost credit and device specific event types. + */ + switch (cgn_evt) { + case FPIN_CONGN_LOST_CREDIT: + nm_log = true; + break; + case FPIN_CONGN_CREDIT_STALL: + nm_log = true; + fallthrough; + case FPIN_CONGN_OVERSUBSCRIPTION: + if (cgn_evt == FPIN_CONGN_OVERSUBSCRIPTION) + nm_log = false; + switch (cgn_sev) { + case FPIN_CONGN_SEVERITY_ERROR: + /* Take action here for an Alarm event */ + break; + case FPIN_CONGN_SEVERITY_WARNING: + /* Take action here for a Warning event */ + break; + } + break; + } + + /* Change the log level to unmaskable for the following event types. */ + lpfc_printf_log(phba, (nm_log ? KERN_WARNING : KERN_INFO), + LOG_CGN_MGMT | LOG_ELS, + "4683 FPIN CONGESTION %s type %s (x%x) Event " + "Duration %d mSecs\n", + cgn_sev_str, cgn_evt_str, cgn_evt, + be32_to_cpu(cgn->event_period)); + return rc; +} + +static void +lpfc_els_rcv_fpin(struct lpfc_vport *vport, void *p, u32 fpin_length) +{ + struct lpfc_hba *phba = vport->phba; + struct fc_els_fpin *fpin = (struct fc_els_fpin *)p; + struct fc_tlv_desc *tlv, *first_tlv, *current_tlv; const char *dtag_nm; - uint32_t desc_cnt = 0, bytes_remain; - u32 dtag; + int desc_cnt = 0, bytes_remain, cnt; + u32 dtag, deliver = 0; + int len; /* FPINs handled only if we are in the right discovery state */ if (vport->port_state < LPFC_DISC_AUTH) @@ -8740,35 +8951,91 @@ lpfc_els_rcv_fpin(struct lpfc_vport *vport, struct fc_els_fpin *fpin, if (fpin_length < sizeof(struct fc_els_fpin)) return; + /* Sanity check descriptor length. The desc_len value does not + * include space for the ELS command and the desc_len fields. + */ + len = be32_to_cpu(fpin->desc_len); + if (fpin_length < len + sizeof(struct fc_els_fpin)) { + lpfc_printf_log(phba, KERN_WARNING, LOG_CGN_MGMT, + "4671 Bad ELS FPIN length %d: %d\n", + len, fpin_length); + return; + } + tlv = (struct fc_tlv_desc *)&fpin->fpin_desc[0]; + first_tlv = tlv; bytes_remain = fpin_length - offsetof(struct fc_els_fpin, fpin_desc); bytes_remain = min_t(u32, bytes_remain, be32_to_cpu(fpin->desc_len)); - /* process each descriptor */ + /* process each descriptor separately */ while (bytes_remain >= FC_TLV_DESC_HDR_SZ && bytes_remain >= FC_TLV_DESC_SZ_FROM_LENGTH(tlv)) { - dtag = be32_to_cpu(tlv->desc_tag); switch (dtag) { case ELS_DTAG_LNK_INTEGRITY: - lpfc_els_rcv_fpin_li(vport, tlv); + lpfc_els_rcv_fpin_li(phba, tlv); + deliver = 1; + break; + case ELS_DTAG_DELIVERY: + lpfc_els_rcv_fpin_del(phba, tlv); + deliver = 1; + break; + case ELS_DTAG_PEER_CONGEST: + lpfc_els_rcv_fpin_peer_cgn(phba, tlv); + deliver = 1; + break; + case ELS_DTAG_CONGESTION: + deliver = lpfc_els_rcv_fpin_cgn(phba, tlv); break; default: dtag_nm = lpfc_get_tlv_dtag_nm(dtag); - lpfc_printf_vlog(vport, KERN_ERR, LOG_TRACE_EVENT, - "4678 skipped FPIN descriptor[%d]: " - "tag x%x (%s)\n", - desc_cnt, dtag, dtag_nm); - break; + lpfc_printf_log(phba, KERN_WARNING, LOG_CGN_MGMT, + "4678 unknown FPIN descriptor[%d]: " + "tag x%x (%s)\n", + desc_cnt, dtag, dtag_nm); + + /* If descriptor is bad, drop the rest of the data */ + return; } + cnt = be32_to_cpu(tlv->desc_len); - desc_cnt++; + /* Sanity check descriptor length. The desc_len value does not + * include space for the desc_tag and the desc_len fields. + */ + len -= (cnt + sizeof(struct fc_tlv_desc)); + if (len < 0) { + dtag_nm = lpfc_get_tlv_dtag_nm(dtag); + lpfc_printf_log(phba, KERN_WARNING, LOG_CGN_MGMT, + "4672 Bad FPIN descriptor TLV length " + "%d: %d %d %s\n", + cnt, len, fpin_length, dtag_nm); + return; + } + + current_tlv = tlv; bytes_remain -= FC_TLV_DESC_SZ_FROM_LENGTH(tlv); tlv = fc_tlv_next_desc(tlv); - } - fc_host_fpin_rcv(lpfc_shost_from_vport(vport), fpin_length, - (char *)fpin); + /* Format payload such that the FPIN delivered to the + * upper layer is a single descriptor FPIN. + */ + if (desc_cnt) + memcpy(first_tlv, current_tlv, + (cnt + sizeof(struct fc_els_fpin))); + + /* Adjust the length so that it only reflects a + * single descriptor FPIN. + */ + fpin_length = cnt + sizeof(struct fc_els_fpin); + fpin->desc_len = cpu_to_be32(fpin_length); + fpin_length += sizeof(struct fc_els_fpin); /* the entire FPIN */ + + /* Send every descriptor individually to the upper layer */ + if (deliver) + fc_host_fpin_rcv(lpfc_shost_from_vport(vport), + fpin_length, (char *)fpin); + desc_cnt++; + } } /** diff --git a/drivers/scsi/lpfc/lpfc_hw4.h b/drivers/scsi/lpfc/lpfc_hw4.h index fdc22e5d5fac..65bb4a66ccf0 100644 --- a/drivers/scsi/lpfc/lpfc_hw4.h +++ b/drivers/scsi/lpfc/lpfc_hw4.h @@ -20,6 +20,7 @@ * included with this package. * *******************************************************************/ +#include #include /* Macros to deal with bit fields. Each bit field must have 3 #defines @@ -4813,3 +4814,16 @@ struct lpfc_grp_hdr { #define LPFC_FW_DUMP 1 #define LPFC_FW_RESET 2 #define LPFC_DV_RESET 3 + +/* + * Initializer useful for decoding FPIN string table. + */ +#define FC_FPIN_CONGN_SEVERITY_INIT { \ + { FPIN_CONGN_SEVERITY_WARNING, "Warning" }, \ + { FPIN_CONGN_SEVERITY_ERROR, "Alarm" }, \ +} + +/* Used for logging FPIN messages */ +#define LPFC_FPIN_WWPN_LINE_SZ 128 +#define LPFC_FPIN_WWPN_LINE_CNT 6 +#define LPFC_FPIN_WWPN_NUM_LINE 6 From patchwork Fri Aug 13 23:00:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 496859 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7BD01C4338F for ; Fri, 13 Aug 2021 23:01:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5C6EC60FC4 for ; Fri, 13 Aug 2021 23:01:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235414AbhHMXBc (ORCPT ); Fri, 13 Aug 2021 19:01:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41552 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235368AbhHMXBc (ORCPT ); Fri, 13 Aug 2021 19:01:32 -0400 Received: from mail-pj1-x102b.google.com (mail-pj1-x102b.google.com [IPv6:2607:f8b0:4864:20::102b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 460D9C061756 for ; Fri, 13 Aug 2021 16:01:04 -0700 (PDT) Received: by mail-pj1-x102b.google.com with SMTP id u13-20020a17090abb0db0290177e1d9b3f7so22929469pjr.1 for ; Fri, 13 Aug 2021 16:01:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kKN3zCQZvHL8YTVbh7xjWWQ0mgqMlRBldUHgdOdkfOc=; b=g31cpC/+sISgiT+wF8rrx+d+KD6Lna5FkijMFk9y3N/z8zBVAj2/Vrpw+b919Bh9yf FKSRnoaWmPY7+7Gl0i7XQQs9vFPT+oSYAtOUIbouGq1/2bDUDKlP9wcKwJCC+yuQm9fM ESSD2ocB4rmNtgkJsfhh7Ya1tfFb+FgZaZAoQxV51f/cMW/9f3IrQ9v6YiDpBag869UZ bsMnH3XzVdo3CIM7pTRTs4JbLp68dlvJTi/mkRiqQAyjSVSrLuf7m++fY2YvCgQ0utCW CzPBHX1Dpypu4M/fpOX59IoR1nxjavbK6BJ+Y3oUiSHToO9vmbY12jmKO8FSwte0EumR 3O5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kKN3zCQZvHL8YTVbh7xjWWQ0mgqMlRBldUHgdOdkfOc=; b=r9sU3cngLZHpff262y5G1pmrOjVmyjNiBUyzFpaIgjtQcm9r38Jd1KlK0BaTgHabHP Iy6cp01f8icQFUs1nIdT/zQMzhfff49tsWDmMe0GVwKhhLFiaBAjBzOel+4ClpXq9GEJ tNATmFaC6oEOMF4jHf/Gh+sFe0YMJNY8Xk38+TyHhGVuT+6b3ZMfvJW4n+ihUgNhif7q pQvi6K3QSv6DWjlqumVnSS4k8NloPgqzDdogDuP43DdBVrhdNiSdTzFQSQcPQikMuzRd tXuJ+VpF3gODslkaiuj5jskDw8rtO2Pk0itIlC5TtpNuPE1QyAtgoyaAT79EHcp77vVM nOZg== X-Gm-Message-State: AOAM532qWeoNQvfxf815ktaFCac3J/OSXUf5kMJwjIBOnANJ/ajs4Ixm Z47hxDUfUCRenFNe5e4X0EG3kWvJ1H4= X-Google-Smtp-Source: ABdhPJxHH/6V8AXqmwWxnqtXF21VG/f82S44amN7VI10agUd/FpPbcOFY2svvVErjodoxfdaMsshWg== X-Received: by 2002:a17:902:bc41:b029:12d:3f9b:401e with SMTP id t1-20020a170902bc41b029012d3f9b401emr3835537plz.47.1628895660966; Fri, 13 Aug 2021 16:01:00 -0700 (PDT) Received: from localhost.localdomain.localdomain ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id e8sm4001997pgg.31.2021.08.13.16.00.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 16:01:00 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH v2 05/16] lpfc: Add EDC ELS support Date: Fri, 13 Aug 2021 16:00:28 -0700 Message-Id: <20210813230039.110546-6-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210813230039.110546-1-jsmart2021@gmail.com> References: <20210813230039.110546-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org When congestion management is enabled, issue EDC ELS to register congestion signaling capabilities with the fabric. The response handling will process the fabric parameters and set the reporting parameters. Similarly, add support for receiving an EDC request from the fabric generating a corresponding response. Implement handlers for congestion signals from the fabric and maintain statistics for them. Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- v2: Addressed kernel test robot warnings for printk arg types. Substituted 0x%zx for %ld for sizeof args. --- drivers/scsi/lpfc/lpfc.h | 35 ++ drivers/scsi/lpfc/lpfc_attr.c | 27 ++ drivers/scsi/lpfc/lpfc_crtn.h | 6 + drivers/scsi/lpfc/lpfc_ct.c | 2 + drivers/scsi/lpfc/lpfc_els.c | 637 ++++++++++++++++++++++++++++++- drivers/scsi/lpfc/lpfc_hbadisc.c | 19 +- drivers/scsi/lpfc/lpfc_hw.h | 2 + drivers/scsi/lpfc/lpfc_hw4.h | 58 +++ drivers/scsi/lpfc/lpfc_init.c | 86 ++++- drivers/scsi/lpfc/lpfc_sli.c | 127 +++++- 10 files changed, 988 insertions(+), 11 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h index dd3ddfa5f761..f23905b89ee3 100644 --- a/drivers/scsi/lpfc/lpfc.h +++ b/drivers/scsi/lpfc/lpfc.h @@ -403,6 +403,11 @@ struct lpfc_trunk_link { link3; }; +struct lpfc_cgn_acqe_stat { + atomic64_t alarm; + atomic64_t warn; +}; + struct lpfc_vport { struct lpfc_hba *phba; struct list_head listentry; @@ -1343,6 +1348,36 @@ struct lpfc_hba { uint64_t ktime_seg10_min; uint64_t ktime_seg10_max; #endif + /* CMF objects */ + u32 cmf_active_mode; +#define LPFC_CFG_OFF 0 + + /* Signal / FPIN handling for Congestion Mgmt */ + u8 cgn_reg_fpin; /* Negotiated value from RDF */ + u8 cgn_init_reg_fpin; /* Initial value from READ_CONFIG */ +#define LPFC_CGN_FPIN_NONE 0x0 +#define LPFC_CGN_FPIN_WARN 0x1 +#define LPFC_CGN_FPIN_ALARM 0x2 +#define LPFC_CGN_FPIN_BOTH (LPFC_CGN_FPIN_WARN | LPFC_CGN_FPIN_ALARM) + + u8 cgn_reg_signal; /* Negotiated value from EDC */ + u8 cgn_init_reg_signal; /* Initial value from READ_CONFIG */ + /* cgn_reg_signal and cgn_init_reg_signal use + * enum fc_edc_cg_signal_cap_types + */ + u16 cgn_fpin_frequency; +#define LPFC_FPIN_INIT_FREQ 0xffff + u32 cgn_sig_freq; + u32 cgn_acqe_cnt; + + /* Statistics counter for ACQE cgn alarms and warnings */ + struct lpfc_cgn_acqe_stat cgn_acqe_stat; + + /* Congestion buffer information */ + atomic_t cgn_fabric_warn_cnt; /* Total warning cgn events for info */ + atomic_t cgn_fabric_alarm_cnt; /* Total alarm cgn events for info */ + atomic_t cgn_sync_warn_cnt; /* Total warning events for SYNC wqe */ + atomic_t cgn_sync_alarm_cnt; /* Total alarm events for SYNC wqe */ struct hlist_node cpuhp; /* used for cpuhp per hba callback */ struct timer_list cpuhp_poll_timer; diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c index 869c2b6f1515..d16d3544084f 100644 --- a/drivers/scsi/lpfc/lpfc_attr.c +++ b/drivers/scsi/lpfc/lpfc_attr.c @@ -6150,6 +6150,19 @@ LPFC_ATTR_RW(ras_fwlog_func, 0, 0, 7, "Firmware Logging Enabled on Function"); */ LPFC_BBCR_ATTR_RW(enable_bbcr, 1, 0, 1, "Enable BBC Recovery"); +/* Signaling module parameters */ +int lpfc_fabric_cgn_frequency = 100; /* 100 ms default */ +module_param(lpfc_fabric_cgn_frequency, int, 0444); +MODULE_PARM_DESC(lpfc_fabric_cgn_frequency, "Congestion signaling fabric freq"); + +int lpfc_acqe_cgn_frequency = 10; /* 10 sec default */ +module_param(lpfc_acqe_cgn_frequency, int, 0444); +MODULE_PARM_DESC(lpfc_acqe_cgn_frequency, "Congestion signaling ACQE freq"); + +int lpfc_use_cgn_signal = 1; /* 0 - only use FPINs, 1 - Use signals if avail */ +module_param(lpfc_use_cgn_signal, int, 0444); +MODULE_PARM_DESC(lpfc_use_cgn_signal, "Use Congestion signaling if available"); + /* * lpfc_enable_dpp: Enable DPP on G7 * 0 = DPP on G7 disabled @@ -6915,6 +6928,9 @@ lpfc_get_stats(struct Scsi_Host *shost) hs->invalid_crc_count = pmb->un.varRdLnk.crcCnt; hs->error_frames = pmb->un.varRdLnk.crcCnt; + hs->cn_sig_warn = atomic64_read(&phba->cgn_acqe_stat.warn); + hs->cn_sig_alarm = atomic64_read(&phba->cgn_acqe_stat.alarm); + hs->link_failure_count -= lso->link_failure_count; hs->loss_of_sync_count -= lso->loss_of_sync_count; hs->loss_of_signal_count -= lso->loss_of_signal_count; @@ -7026,6 +7042,12 @@ lpfc_reset_stats(struct Scsi_Host *shost) else lso->link_events = (phba->fc_eventTag >> 1); + atomic64_set(&phba->cgn_acqe_stat.warn, 0); + atomic64_set(&phba->cgn_acqe_stat.alarm, 0); + + memset(&shost_to_fc_host(shost)->fpin_stats, 0, + sizeof(shost_to_fc_host(shost)->fpin_stats)); + psli->stats_start = ktime_get_seconds(); mempool_free(pmboxq, phba->mbox_mem_pool); @@ -7459,6 +7481,11 @@ lpfc_get_cfgparam(struct lpfc_hba *phba) lpfc_enable_dpp_init(phba, lpfc_enable_dpp); lpfc_enable_mi_init(phba, lpfc_enable_mi); + phba->cmf_active_mode = LPFC_CFG_OFF; + if (lpfc_fabric_cgn_frequency > EDC_CG_SIGFREQ_CNT_MAX || + lpfc_fabric_cgn_frequency < EDC_CG_SIGFREQ_CNT_MIN) + lpfc_fabric_cgn_frequency = 100; /* 100 ms default */ + if (phba->sli_rev != LPFC_SLI_REV4) { /* NVME only supported on SLI4 */ phba->nvmet_support = 0; diff --git a/drivers/scsi/lpfc/lpfc_crtn.h b/drivers/scsi/lpfc/lpfc_crtn.h index 41e0d8ef015a..b1db01884990 100644 --- a/drivers/scsi/lpfc/lpfc_crtn.h +++ b/drivers/scsi/lpfc/lpfc_crtn.h @@ -74,6 +74,7 @@ int lpfc_init_iocb_list(struct lpfc_hba *phba, int cnt); void lpfc_free_iocb_list(struct lpfc_hba *phba); int lpfc_post_rq_buffer(struct lpfc_hba *phba, struct lpfc_queue *hrq, struct lpfc_queue *drq, int count, int idx); +int lpfc_config_cgn_signal(struct lpfc_hba *phba); void lpfc_mbx_cmpl_local_config_link(struct lpfc_hba *, LPFC_MBOXQ_t *); void lpfc_mbx_cmpl_reg_login(struct lpfc_hba *, LPFC_MBOXQ_t *); @@ -143,6 +144,7 @@ int lpfc_issue_els_scr(struct lpfc_vport *vport, uint8_t retry); int lpfc_issue_els_rscn(struct lpfc_vport *vport, uint8_t retry); int lpfc_issue_fabric_reglogin(struct lpfc_vport *); int lpfc_issue_els_rdf(struct lpfc_vport *vport, uint8_t retry); +int lpfc_issue_els_edc(struct lpfc_vport *vport, uint8_t retry); int lpfc_els_free_iocb(struct lpfc_hba *, struct lpfc_iocbq *); int lpfc_ct_free_iocb(struct lpfc_hba *, struct lpfc_iocbq *); int lpfc_els_rsp_acc(struct lpfc_vport *, uint32_t, struct lpfc_iocbq *, @@ -607,6 +609,10 @@ extern int lpfc_enable_nvmet_cnt; extern unsigned long long lpfc_enable_nvmet[]; extern int lpfc_no_hba_reset_cnt; extern unsigned long lpfc_no_hba_reset[]; +extern int lpfc_acqe_cgn_frequency; +extern int lpfc_fabric_cgn_frequency; +extern int lpfc_use_cgn_signal; + extern union lpfc_wqe128 lpfc_iread_cmd_template; extern union lpfc_wqe128 lpfc_iwrite_cmd_template; extern union lpfc_wqe128 lpfc_icmnd_cmd_template; diff --git a/drivers/scsi/lpfc/lpfc_ct.c b/drivers/scsi/lpfc/lpfc_ct.c index 435349f893ad..dfcb7d4bd7fa 100644 --- a/drivers/scsi/lpfc/lpfc_ct.c +++ b/drivers/scsi/lpfc/lpfc_ct.c @@ -2288,6 +2288,8 @@ lpfc_cmpl_ct_disc_fdmi(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, /* No retry on Vendor, RPA only done on physical port */ if (phba->link_flag & LS_CT_VEN_RPA) { phba->link_flag &= ~LS_CT_VEN_RPA; + if (phba->cmf_active_mode == LPFC_CFG_OFF) + return; lpfc_printf_log(phba, KERN_ERR, LOG_DISCOVERY | LOG_ELS, "6460 VEN FDMI RPA failure\n"); diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c index 097f132fc35b..3b1b0d3ff0d9 100644 --- a/drivers/scsi/lpfc/lpfc_els.c +++ b/drivers/scsi/lpfc/lpfc_els.c @@ -56,6 +56,9 @@ static int lpfc_issue_els_fdisc(struct lpfc_vport *vport, struct lpfc_nodelist *ndlp, uint8_t retry); static int lpfc_issue_fabric_iocb(struct lpfc_hba *phba, struct lpfc_iocbq *iocb); +static void lpfc_cmpl_els_edc(struct lpfc_hba *phba, + struct lpfc_iocbq *cmdiocb, + struct lpfc_iocbq *rspiocb); static void lpfc_cmpl_els_uvem(struct lpfc_hba *, struct lpfc_iocbq *, struct lpfc_iocbq *); @@ -3286,6 +3289,9 @@ lpfc_cmpl_els_disc_cmd(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, case ELS_CMD_SCR: lpfc_issue_els_scr(vport, cmdiocb->retry); break; + case ELS_CMD_EDC: + lpfc_issue_els_edc(vport, cmdiocb->retry); + break; case ELS_CMD_RDF: cmdiocb->context1 = NULL; /* save ndlp refcnt */ lpfc_issue_els_rdf(vport, cmdiocb->retry); @@ -3295,6 +3301,11 @@ lpfc_cmpl_els_disc_cmd(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, } phba->fc_stat.elsRetryExceeded++; } + if (cmd == ELS_CMD_EDC) { + /* must be called before checking uplStatus and returning */ + lpfc_cmpl_els_edc(phba, cmdiocb, rspiocb); + return; + } if (irsp->ulpStatus) { /* ELS discovery cmd completes with error */ lpfc_printf_vlog(vport, KERN_WARNING, LOG_ELS, @@ -3751,6 +3762,425 @@ lpfc_els_rcv_rdf(struct lpfc_vport *vport, struct lpfc_iocbq *cmdiocb, return 0; } +/** + * lpfc_least_capable_settings - helper function for EDC rsp processing + * @phba: pointer to lpfc hba data structure. + * @pcgd: pointer to congestion detection descriptor in EDC rsp. + * + * This helper routine determines the least capable setting for + * congestion signals, signal freq, including scale, from the + * congestion detection descriptor in the EDC rsp. The routine + * sets @phba values in preparation for a set_featues mailbox. + **/ +static void +lpfc_least_capable_settings(struct lpfc_hba *phba, + struct fc_diag_cg_sig_desc *pcgd) +{ + u32 rsp_sig_cap = 0, drv_sig_cap = 0; + u32 rsp_sig_freq_cyc = 0, rsp_sig_freq_scale = 0; + + /* Get rsp signal and frequency capabilities. */ + rsp_sig_cap = be32_to_cpu(pcgd->xmt_signal_capability); + rsp_sig_freq_cyc = be16_to_cpu(pcgd->xmt_signal_frequency.count); + rsp_sig_freq_scale = be16_to_cpu(pcgd->xmt_signal_frequency.units); + + /* If the Fport does not support signals. Set FPIN only */ + if (rsp_sig_cap == EDC_CG_SIG_NOTSUPPORTED) + goto out_no_support; + + /* Apply the xmt scale to the xmt cycle to get the correct frequency. + * Adapter default is 100 millisSeconds. Convert all xmt cycle values + * to milliSeconds. + */ + switch (rsp_sig_freq_scale) { + case EDC_CG_SIGFREQ_SEC: + rsp_sig_freq_cyc *= MSEC_PER_SEC; + break; + case EDC_CG_SIGFREQ_MSEC: + rsp_sig_freq_cyc = 1; + break; + default: + goto out_no_support; + } + + /* Convenient shorthand. */ + drv_sig_cap = phba->cgn_reg_signal; + + /* Choose the least capable frequency. */ + if (rsp_sig_freq_cyc > phba->cgn_sig_freq) + phba->cgn_sig_freq = rsp_sig_freq_cyc; + + /* Should be some common signals support. Settle on least capable + * signal and adjust FPIN values. Initialize defaults to ease the + * decision. + */ + phba->cgn_reg_fpin = LPFC_CGN_FPIN_WARN | LPFC_CGN_FPIN_ALARM; + phba->cgn_reg_signal = EDC_CG_SIG_NOTSUPPORTED; + if (rsp_sig_cap == EDC_CG_SIG_WARN_ONLY && + (drv_sig_cap == EDC_CG_SIG_WARN_ONLY || + drv_sig_cap == EDC_CG_SIG_WARN_ALARM)) { + phba->cgn_reg_signal = EDC_CG_SIG_WARN_ONLY; + phba->cgn_reg_fpin &= ~LPFC_CGN_FPIN_WARN; + } + if (rsp_sig_cap == EDC_CG_SIG_WARN_ALARM) { + if (drv_sig_cap == EDC_CG_SIG_WARN_ALARM) { + phba->cgn_reg_signal = EDC_CG_SIG_WARN_ALARM; + phba->cgn_reg_fpin = LPFC_CGN_FPIN_NONE; + } + if (drv_sig_cap == EDC_CG_SIG_WARN_ONLY) { + phba->cgn_reg_signal = EDC_CG_SIG_WARN_ONLY; + phba->cgn_reg_fpin &= ~LPFC_CGN_FPIN_WARN; + } + } + return; + +out_no_support: + phba->cgn_reg_signal = EDC_CG_SIG_NOTSUPPORTED; + phba->cgn_sig_freq = 0; + phba->cgn_reg_fpin = LPFC_CGN_FPIN_ALARM | LPFC_CGN_FPIN_WARN; +} + +DECLARE_ENUM2STR_LOOKUP(lpfc_get_tlv_dtag_nm, fc_ls_tlv_dtag, + FC_LS_TLV_DTAG_INIT); + +/** + * lpfc_cmpl_els_edc - Completion callback function for EDC + * @phba: pointer to lpfc hba data structure. + * @cmdiocb: pointer to lpfc command iocb data structure. + * @rspiocb: pointer to lpfc response iocb data structure. + * + * This routine is the completion callback function for issuing the Exchange + * Diagnostic Capabilities (EDC) command. The driver issues an EDC to + * notify the FPort of its Congestion and Link Fault capabilities. This + * routine parses the FPort's response and decides on the least common + * values applicable to both FPort and NPort for Warnings and Alarms that + * are communicated via hardware signals. + **/ +static void +lpfc_cmpl_els_edc(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, + struct lpfc_iocbq *rspiocb) +{ + IOCB_t *irsp; + struct fc_els_edc_resp *edc_rsp; + struct fc_tlv_desc *tlv; + struct fc_diag_cg_sig_desc *pcgd; + struct fc_diag_lnkflt_desc *plnkflt; + struct lpfc_dmabuf *pcmd, *prsp; + const char *dtag_nm; + u32 *pdata, dtag; + int desc_cnt = 0, bytes_remain; + bool rcv_cap_desc = false; + struct lpfc_nodelist *ndlp; + + irsp = &rspiocb->iocb; + ndlp = cmdiocb->context1; + + lpfc_debugfs_disc_trc(phba->pport, LPFC_DISC_TRC_ELS_CMD, + "EDC cmpl: status:x%x/x%x did:x%x", + irsp->ulpStatus, irsp->un.ulpWord[4], + irsp->un.elsreq64.remoteID); + + /* ELS cmd tag completes */ + lpfc_printf_log(phba, KERN_INFO, LOG_ELS | LOG_CGN_MGMT, + "4201 EDC cmd tag x%x completes Data: x%x x%x x%x\n", + irsp->ulpIoTag, irsp->ulpStatus, + irsp->un.ulpWord[4], irsp->ulpTimeout); + + pcmd = (struct lpfc_dmabuf *)cmdiocb->context2; + if (!pcmd) + goto out; + + pdata = (u32 *)pcmd->virt; + if (!pdata) + goto out; + + /* Need to clear signal values, send features MB and RDF with FPIN. */ + if (irsp->ulpStatus) + goto out; + + prsp = list_get_first(&pcmd->list, struct lpfc_dmabuf, list); + if (!prsp) + goto out; + + edc_rsp = prsp->virt; + if (!edc_rsp) + goto out; + + /* ELS cmd tag completes */ + lpfc_printf_log(phba, KERN_INFO, LOG_ELS | LOG_CGN_MGMT, + "4676 Fabric EDC Rsp: " + "0x%02x, 0x%08x\n", + edc_rsp->acc_hdr.la_cmd, + be32_to_cpu(edc_rsp->desc_list_len)); + + /* + * Payload length in bytes is the response descriptor list + * length minus the 12 bytes of Link Service Request + * Information descriptor in the reply. + */ + bytes_remain = be32_to_cpu(edc_rsp->desc_list_len) - + sizeof(struct fc_els_lsri_desc); + if (bytes_remain <= 0) + goto out; + + tlv = edc_rsp->desc; + + /* + * cycle through EDC diagnostic descriptors to find the + * congestion signaling capability descriptor + */ + while (bytes_remain) { + if (bytes_remain < FC_TLV_DESC_HDR_SZ) { + lpfc_printf_log(phba, KERN_WARNING, LOG_CGN_MGMT, + "6461 Truncated TLV hdr on " + "Diagnostic descriptor[%d]\n", + desc_cnt); + goto out; + } + + dtag = be32_to_cpu(tlv->desc_tag); + switch (dtag) { + case ELS_DTAG_LNK_FAULT_CAP: + if (bytes_remain < FC_TLV_DESC_SZ_FROM_LENGTH(tlv) || + FC_TLV_DESC_SZ_FROM_LENGTH(tlv) != + sizeof(struct fc_diag_lnkflt_desc)) { + lpfc_printf_log( + phba, KERN_WARNING, LOG_CGN_MGMT, + "6462 Truncated Link Fault Diagnostic " + "descriptor[%d]: %d vs 0x%zx 0x%zx\n", + desc_cnt, bytes_remain, + FC_TLV_DESC_SZ_FROM_LENGTH(tlv), + sizeof(struct fc_diag_cg_sig_desc)); + goto out; + } + plnkflt = (struct fc_diag_lnkflt_desc *)tlv; + lpfc_printf_log( + phba, KERN_INFO, LOG_ELS | LOG_CGN_MGMT, + "4617 Link Fault Desc Data: 0x%08x 0x%08x " + "0x%08x 0x%08x 0x%08x\n", + be32_to_cpu(plnkflt->desc_tag), + be32_to_cpu(plnkflt->desc_len), + be32_to_cpu( + plnkflt->degrade_activate_threshold), + be32_to_cpu( + plnkflt->degrade_deactivate_threshold), + be32_to_cpu(plnkflt->fec_degrade_interval)); + break; + case ELS_DTAG_CG_SIGNAL_CAP: + if (bytes_remain < FC_TLV_DESC_SZ_FROM_LENGTH(tlv) || + FC_TLV_DESC_SZ_FROM_LENGTH(tlv) != + sizeof(struct fc_diag_cg_sig_desc)) { + lpfc_printf_log( + phba, KERN_WARNING, LOG_CGN_MGMT, + "6463 Truncated Cgn Signal Diagnostic " + "descriptor[%d]: %d vs 0x%zx 0x%zx\n", + desc_cnt, bytes_remain, + FC_TLV_DESC_SZ_FROM_LENGTH(tlv), + sizeof(struct fc_diag_cg_sig_desc)); + goto out; + } + + pcgd = (struct fc_diag_cg_sig_desc *)tlv; + lpfc_printf_log( + phba, KERN_INFO, LOG_ELS | LOG_CGN_MGMT, + "4616 CGN Desc Data: 0x%08x 0x%08x " + "0x%08x 0x%04x 0x%04x 0x%08x 0x%04x 0x%04x\n", + be32_to_cpu(pcgd->desc_tag), + be32_to_cpu(pcgd->desc_len), + be32_to_cpu(pcgd->xmt_signal_capability), + be32_to_cpu(pcgd->xmt_signal_frequency.count), + be32_to_cpu(pcgd->xmt_signal_frequency.units), + be32_to_cpu(pcgd->rcv_signal_capability), + be32_to_cpu(pcgd->rcv_signal_frequency.count), + be32_to_cpu(pcgd->rcv_signal_frequency.units)); + + /* Compare driver and Fport capabilities and choose + * least common. + */ + lpfc_least_capable_settings(phba, pcgd); + rcv_cap_desc = true; + break; + default: + dtag_nm = lpfc_get_tlv_dtag_nm(dtag); + lpfc_printf_log(phba, KERN_WARNING, LOG_CGN_MGMT, + "4919 unknown Diagnostic " + "Descriptor[%d]: tag x%x (%s)\n", + desc_cnt, dtag, dtag_nm); + } + + bytes_remain -= FC_TLV_DESC_SZ_FROM_LENGTH(tlv); + tlv = fc_tlv_next_desc(tlv); + desc_cnt++; + } + +out: + if (!rcv_cap_desc) { + phba->cgn_reg_fpin = LPFC_CGN_FPIN_ALARM | LPFC_CGN_FPIN_WARN; + phba->cgn_reg_signal = EDC_CG_SIG_NOTSUPPORTED; + phba->cgn_sig_freq = 0; + lpfc_printf_log(phba, KERN_WARNING, LOG_ELS | LOG_CGN_MGMT, + "4202 EDC rsp error - sending RDF " + "for FPIN only.\n"); + } + + lpfc_config_cgn_signal(phba); + + /* Check to see if link went down during discovery */ + lpfc_els_chk_latt(phba->pport); + lpfc_debugfs_disc_trc(phba->pport, LPFC_DISC_TRC_ELS_CMD, + "EDC Cmpl: did:x%x refcnt %d", + ndlp->nlp_DID, kref_read(&ndlp->kref), 0); + lpfc_els_free_iocb(phba, cmdiocb); + lpfc_nlp_put(ndlp); +} + +static void +lpfc_format_edc_cgn_desc(struct lpfc_hba *phba, struct fc_diag_cg_sig_desc *cgd) +{ + /* We are assuming cgd was zero'ed before calling this routine */ + + /* Configure the congestion detection capability */ + cgd->desc_tag = cpu_to_be32(ELS_DTAG_CG_SIGNAL_CAP); + + /* Descriptor len doesn't include the tag or len fields. */ + cgd->desc_len = cpu_to_be32( + FC_TLV_DESC_LENGTH_FROM_SZ(struct fc_diag_cg_sig_desc)); + + /* xmt_signal_capability already set to EDC_CG_SIG_NOTSUPPORTED. + * xmt_signal_frequency.count already set to 0. + * xmt_signal_frequency.units already set to 0. + */ + + if (phba->cmf_active_mode == LPFC_CFG_OFF) { + /* rcv_signal_capability already set to EDC_CG_SIG_NOTSUPPORTED. + * rcv_signal_frequency.count already set to 0. + * rcv_signal_frequency.units already set to 0. + */ + phba->cgn_sig_freq = 0; + return; + } + switch (phba->cgn_reg_signal) { + case EDC_CG_SIG_WARN_ONLY: + cgd->rcv_signal_capability = cpu_to_be32(EDC_CG_SIG_WARN_ONLY); + break; + case EDC_CG_SIG_WARN_ALARM: + cgd->rcv_signal_capability = cpu_to_be32(EDC_CG_SIG_WARN_ALARM); + break; + default: + /* rcv_signal_capability left 0 thus no support */ + break; + } + + /* We start negotiation with lpfc_fabric_cgn_frequency, after + * the completion we settle on the higher frequency. + */ + cgd->rcv_signal_frequency.count = + cpu_to_be16(lpfc_fabric_cgn_frequency); + cgd->rcv_signal_frequency.units = + cpu_to_be16(EDC_CG_SIGFREQ_MSEC); +} + + /** + * lpfc_issue_els_edc - Exchange Diagnostic Capabilities with the fabric. + * @vport: pointer to a host virtual N_Port data structure. + * @retry: retry counter for the command iocb. + * + * This routine issues an ELS EDC to the F-Port Controller to communicate + * this N_Port's support of hardware signals in its Congestion + * Capabilities Descriptor. + * + * Note: This routine does not check if one or more signals are + * set in the cgn_reg_signal parameter. The caller makes the + * decision to enforce cgn_reg_signal as nonzero or zero depending + * on the conditions. During Fabric requests, the driver + * requires cgn_reg_signals to be nonzero. But a dynamic request + * to set the congestion mode to OFF from Monitor or Manage + * would correctly issue an EDC with no signals enabled to + * turn off switch functionality and then update the FW. + * + * Return code + * 0 - Successfully issued edc command + * 1 - Failed to issue edc command + **/ +int +lpfc_issue_els_edc(struct lpfc_vport *vport, uint8_t retry) +{ + struct lpfc_hba *phba = vport->phba; + struct lpfc_iocbq *elsiocb; + struct lpfc_els_edc_req *edc_req; + struct fc_diag_cg_sig_desc *cgn_desc; + u16 cmdsize; + struct lpfc_nodelist *ndlp; + u8 *pcmd = NULL; + u32 edc_req_size, cgn_desc_size; + int rc; + + if (vport->port_type == LPFC_NPIV_PORT) + return -EACCES; + + ndlp = lpfc_findnode_did(vport, Fabric_DID); + if (!ndlp || ndlp->nlp_state != NLP_STE_UNMAPPED_NODE) + return -ENODEV; + + /* If HBA doesn't support signals, drop into RDF */ + if (!phba->cgn_init_reg_signal) + goto try_rdf; + + edc_req_size = sizeof(struct fc_els_edc); + cgn_desc_size = sizeof(struct fc_diag_cg_sig_desc); + cmdsize = edc_req_size + cgn_desc_size; + elsiocb = lpfc_prep_els_iocb(vport, 1, cmdsize, retry, ndlp, + ndlp->nlp_DID, ELS_CMD_EDC); + if (!elsiocb) + goto try_rdf; + + /* Configure the payload for the supported Diagnostics capabilities. */ + pcmd = (u8 *)(((struct lpfc_dmabuf *)elsiocb->context2)->virt); + memset(pcmd, 0, cmdsize); + edc_req = (struct lpfc_els_edc_req *)pcmd; + edc_req->edc.desc_len = cpu_to_be32(cgn_desc_size); + edc_req->edc.edc_cmd = ELS_EDC; + + cgn_desc = &edc_req->cgn_desc; + + lpfc_format_edc_cgn_desc(phba, cgn_desc); + + phba->cgn_sig_freq = lpfc_fabric_cgn_frequency; + + lpfc_printf_vlog(vport, KERN_INFO, LOG_CGN_MGMT, + "4623 Xmit EDC to remote " + "NPORT x%x reg_sig x%x reg_fpin:x%x\n", + ndlp->nlp_DID, phba->cgn_reg_signal, + phba->cgn_reg_fpin); + + elsiocb->iocb_cmpl = lpfc_cmpl_els_disc_cmd; + elsiocb->context1 = lpfc_nlp_get(ndlp); + if (!elsiocb->context1) { + lpfc_els_free_iocb(phba, elsiocb); + return -EIO; + } + + lpfc_debugfs_disc_trc(vport, LPFC_DISC_TRC_ELS_CMD, + "Issue EDC: did:x%x refcnt %d", + ndlp->nlp_DID, kref_read(&ndlp->kref), 0); + rc = lpfc_sli_issue_iocb(phba, LPFC_ELS_RING, elsiocb, 0); + if (rc == IOCB_ERROR) { + /* The additional lpfc_nlp_put will cause the following + * lpfc_els_free_iocb routine to trigger the rlease of + * the node. + */ + lpfc_els_free_iocb(phba, elsiocb); + lpfc_nlp_put(ndlp); + goto try_rdf; + } + return 0; +try_rdf: + phba->cgn_reg_fpin = LPFC_CGN_FPIN_WARN | LPFC_CGN_FPIN_ALARM; + phba->cgn_reg_signal = EDC_CG_SIG_NOTSUPPORTED; + rc = lpfc_issue_els_rdf(vport, 0); + return rc; +} + /** * lpfc_cancel_retry_delay_tmo - Cancel the timer with delayed iocb-cmd retry * @vport: pointer to a host virtual N_Port data structure. @@ -4515,7 +4945,7 @@ lpfc_els_free_iocb(struct lpfc_hba *phba, struct lpfc_iocbq *elsiocb) { struct lpfc_dmabuf *buf_ptr, *buf_ptr1; - /* The I/O job is complete. Clear the context1 data. */ + /* The I/O iocb is complete. Clear the context1 data. */ elsiocb->context1 = NULL; /* context2 = cmd, context2->next = rsp, context3 = bpl */ @@ -5162,6 +5592,86 @@ lpfc_els_rsp_reject(struct lpfc_vport *vport, uint32_t rejectError, return 0; } + /** + * lpfc_issue_els_edc_rsp - Exchange Diagnostic Capabilities with the fabric. + * @vport: pointer to a host virtual N_Port data structure. + * @cmdiocb: pointer to the original lpfc command iocb data structure. + * @ndlp: NPort to where rsp is directed + * + * This routine issues an EDC ACC RSP to the F-Port Controller to communicate + * this N_Port's support of hardware signals in its Congestion + * Capabilities Descriptor. + * + * Return code + * 0 - Successfully issued edc rsp command + * 1 - Failed to issue edc rsp command + **/ +static int +lpfc_issue_els_edc_rsp(struct lpfc_vport *vport, struct lpfc_iocbq *cmdiocb, + struct lpfc_nodelist *ndlp) +{ + struct lpfc_hba *phba = vport->phba; + struct lpfc_els_edc_rsp *edc_rsp; + struct lpfc_iocbq *elsiocb; + IOCB_t *icmd, *cmd; + uint8_t *pcmd; + int cmdsize, rc; + + cmdsize = sizeof(struct lpfc_els_edc_rsp); + elsiocb = lpfc_prep_els_iocb(vport, 0, cmdsize, cmdiocb->retry, + ndlp, ndlp->nlp_DID, ELS_CMD_ACC); + if (!elsiocb) + return 1; + + icmd = &elsiocb->iocb; + cmd = &cmdiocb->iocb; + icmd->ulpContext = cmd->ulpContext; /* Xri / rx_id */ + icmd->unsli3.rcvsli3.ox_id = cmd->unsli3.rcvsli3.ox_id; + pcmd = (((struct lpfc_dmabuf *)elsiocb->context2)->virt); + memset(pcmd, 0, cmdsize); + + edc_rsp = (struct lpfc_els_edc_rsp *)pcmd; + edc_rsp->edc_rsp.acc_hdr.la_cmd = ELS_LS_ACC; + edc_rsp->edc_rsp.desc_list_len = cpu_to_be32( + FC_TLV_DESC_LENGTH_FROM_SZ(struct lpfc_els_edc_rsp)); + edc_rsp->edc_rsp.lsri.desc_tag = cpu_to_be32(ELS_DTAG_LS_REQ_INFO); + edc_rsp->edc_rsp.lsri.desc_len = cpu_to_be32( + FC_TLV_DESC_LENGTH_FROM_SZ(struct fc_els_lsri_desc)); + edc_rsp->edc_rsp.lsri.rqst_w0.cmd = ELS_EDC; + lpfc_format_edc_cgn_desc(phba, &edc_rsp->cgn_desc); + + lpfc_debugfs_disc_trc(vport, LPFC_DISC_TRC_ELS_RSP, + "Issue EDC ACC: did:x%x flg:x%x refcnt %d", + ndlp->nlp_DID, ndlp->nlp_flag, + kref_read(&ndlp->kref)); + elsiocb->iocb_cmpl = lpfc_cmpl_els_rsp; + + phba->fc_stat.elsXmitACC++; + elsiocb->context1 = lpfc_nlp_get(ndlp); + if (!elsiocb->context1) { + lpfc_els_free_iocb(phba, elsiocb); + return 1; + } + + rc = lpfc_sli_issue_iocb(phba, LPFC_ELS_RING, elsiocb, 0); + if (rc == IOCB_ERROR) { + lpfc_els_free_iocb(phba, elsiocb); + lpfc_nlp_put(ndlp); + return 1; + } + + /* Xmit ELS ACC response tag */ + lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS, + "0152 Xmit EDC ACC response Status: x%x, IoTag: x%x, " + "XRI: x%x, DID: x%x, nlp_flag: x%x nlp_state: x%x " + "RPI: x%x, fc_flag x%x\n", + rc, elsiocb->iotag, elsiocb->sli4_xritag, + ndlp->nlp_DID, ndlp->nlp_flag, ndlp->nlp_state, + ndlp->nlp_rpi, vport->fc_flag); + + return 0; +} + /** * lpfc_els_rsp_adisc_acc - Prepare and issue acc response to adisc iocb cmd * @vport: pointer to a virtual N_Port data structure. @@ -8231,6 +8741,125 @@ lpfc_els_rcv_fan(struct lpfc_vport *vport, struct lpfc_iocbq *cmdiocb, return 0; } +/** + * lpfc_els_rcv_edc - Process an unsolicited EDC iocb + * @vport: pointer to a host virtual N_Port data structure. + * @cmdiocb: pointer to lpfc command iocb data structure. + * @ndlp: pointer to a node-list data structure. + * + * Return code + * 0 - Successfully processed echo iocb (currently always return 0) + **/ +static int +lpfc_els_rcv_edc(struct lpfc_vport *vport, struct lpfc_iocbq *cmdiocb, + struct lpfc_nodelist *ndlp) +{ + struct lpfc_hba *phba = vport->phba; + struct fc_els_edc *edc_req; + struct fc_tlv_desc *tlv; + uint8_t *payload; + uint32_t *ptr, dtag; + const char *dtag_nm; + int desc_cnt = 0, bytes_remain; + bool rcv_cap_desc = false; + + payload = ((struct lpfc_dmabuf *)cmdiocb->context2)->virt; + + edc_req = (struct fc_els_edc *)payload; + bytes_remain = be32_to_cpu(edc_req->desc_len); + + ptr = (uint32_t *)payload; + lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS | LOG_CGN_MGMT, + "3319 Rcv EDC payload len %d: x%x x%x x%x\n", + bytes_remain, be32_to_cpu(*ptr), + be32_to_cpu(*(ptr + 1)), be32_to_cpu(*(ptr + 2))); + + /* No signal support unless there is a congestion descriptor */ + phba->cgn_reg_signal = EDC_CG_SIG_NOTSUPPORTED; + phba->cgn_sig_freq = 0; + phba->cgn_reg_fpin = LPFC_CGN_FPIN_ALARM | LPFC_CGN_FPIN_WARN; + + if (bytes_remain <= 0) + goto out; + + tlv = edc_req->desc; + + /* + * cycle through EDC diagnostic descriptors to find the + * congestion signaling capability descriptor + */ + while (bytes_remain && !rcv_cap_desc) { + if (bytes_remain < FC_TLV_DESC_HDR_SZ) { + lpfc_printf_log(phba, KERN_WARNING, LOG_CGN_MGMT, + "6464 Truncated TLV hdr on " + "Diagnostic descriptor[%d]\n", + desc_cnt); + goto out; + } + + dtag = be32_to_cpu(tlv->desc_tag); + switch (dtag) { + case ELS_DTAG_LNK_FAULT_CAP: + if (bytes_remain < FC_TLV_DESC_SZ_FROM_LENGTH(tlv) || + FC_TLV_DESC_SZ_FROM_LENGTH(tlv) != + sizeof(struct fc_diag_lnkflt_desc)) { + lpfc_printf_log( + phba, KERN_WARNING, LOG_CGN_MGMT, + "6465 Truncated Link Fault Diagnostic " + "descriptor[%d]: %d vs 0x%zx 0x%zx\n", + desc_cnt, bytes_remain, + FC_TLV_DESC_SZ_FROM_LENGTH(tlv), + sizeof(struct fc_diag_cg_sig_desc)); + goto out; + } + /* No action for Link Fault descriptor for now */ + break; + case ELS_DTAG_CG_SIGNAL_CAP: + if (bytes_remain < FC_TLV_DESC_SZ_FROM_LENGTH(tlv) || + FC_TLV_DESC_SZ_FROM_LENGTH(tlv) != + sizeof(struct fc_diag_cg_sig_desc)) { + lpfc_printf_log( + phba, KERN_WARNING, LOG_CGN_MGMT, + "6466 Truncated cgn signal Diagnostic " + "descriptor[%d]: %d vs 0x%zx 0x%zx\n", + desc_cnt, bytes_remain, + FC_TLV_DESC_SZ_FROM_LENGTH(tlv), + sizeof(struct fc_diag_cg_sig_desc)); + goto out; + } + + phba->cgn_reg_fpin = phba->cgn_init_reg_fpin; + phba->cgn_reg_signal = phba->cgn_init_reg_signal; + + /* We start negotiation with lpfc_fabric_cgn_frequency. + * When we process the EDC, we will settle on the + * higher frequency. + */ + phba->cgn_sig_freq = lpfc_fabric_cgn_frequency; + + lpfc_least_capable_settings( + phba, (struct fc_diag_cg_sig_desc *)tlv); + rcv_cap_desc = true; + break; + default: + dtag_nm = lpfc_get_tlv_dtag_nm(dtag); + lpfc_printf_log(phba, KERN_WARNING, LOG_CGN_MGMT, + "6467 unknown Diagnostic " + "Descriptor[%d]: tag x%x (%s)\n", + desc_cnt, dtag, dtag_nm); + } + bytes_remain -= FC_TLV_DESC_SZ_FROM_LENGTH(tlv); + tlv = fc_tlv_next_desc(tlv); + desc_cnt++; + } +out: + /* Need to send back an ACC */ + lpfc_issue_els_edc_rsp(vport, cmdiocb, ndlp); + + lpfc_config_cgn_signal(phba); + return 0; +} + /** * lpfc_els_timeout - Handler funciton to the els timer * @t: timer context used to obtain the vport. @@ -8688,9 +9317,6 @@ lpfc_send_els_event(struct lpfc_vport *vport, } -DECLARE_ENUM2STR_LOOKUP(lpfc_get_tlv_dtag_nm, fc_ls_tlv_dtag, - FC_LS_TLV_DTAG_INIT); - DECLARE_ENUM2STR_LOOKUP(lpfc_get_fpin_li_event_nm, fc_fpin_li_event_types, FC_FPIN_LI_EVT_TYPES_INIT); @@ -9426,6 +10052,9 @@ lpfc_els_unsol_buffer(struct lpfc_hba *phba, struct lpfc_sli_ring *pring, /* There are no replies, so no rjt codes */ break; + case ELS_CMD_EDC: + lpfc_els_rcv_edc(vport, elsiocb, ndlp); + break; case ELS_CMD_RDF: phba->fc_stat.elsRcvRDF++; /* Accept RDF only from fabric controller */ diff --git a/drivers/scsi/lpfc/lpfc_hbadisc.c b/drivers/scsi/lpfc/lpfc_hbadisc.c index 6da2daf7d9e3..95989230b47e 100644 --- a/drivers/scsi/lpfc/lpfc_hbadisc.c +++ b/drivers/scsi/lpfc/lpfc_hbadisc.c @@ -4209,6 +4209,7 @@ lpfc_mbx_cmpl_ns_reg_login(struct lpfc_hba *phba, LPFC_MBOXQ_t *pmb) struct lpfc_dmabuf *mp = (struct lpfc_dmabuf *)(pmb->ctx_buf); struct lpfc_nodelist *ndlp = (struct lpfc_nodelist *)pmb->ctx_ndlp; struct lpfc_vport *vport = pmb->vport; + int rc; pmb->ctx_buf = NULL; pmb->ctx_ndlp = NULL; @@ -4284,9 +4285,23 @@ lpfc_mbx_cmpl_ns_reg_login(struct lpfc_hba *phba, LPFC_MBOXQ_t *pmb) /* Issue SCR just before NameServer GID_FT Query */ lpfc_issue_els_scr(vport, 0); - if (!phba->cfg_enable_mi || - phba->sli4_hba.pc_sli4_params.mi_ver < LPFC_MIB3_SUPPORT) + /* Link was bounced or a Fabric LOGO occurred. Start EDC + * with initial FW values provided the congestion mode is + * not off. Note that signals may or may not be supported + * by the adapter but FPIN is provided by default for 1 + * or both missing signals support. + */ + if (phba->cmf_active_mode != LPFC_CFG_OFF) { + phba->cgn_reg_fpin = phba->cgn_init_reg_fpin; + phba->cgn_reg_signal = phba->cgn_init_reg_signal; + rc = lpfc_issue_els_edc(vport, 0); + lpfc_printf_log(phba, KERN_INFO, + LOG_INIT | LOG_ELS | LOG_DISCOVERY, + "4220 EDC issue error x%x, Data: x%x\n", + rc, phba->cgn_init_reg_signal); + } else { lpfc_issue_els_rdf(vport, 0); + } } vport->fc_ns_retry = 0; diff --git a/drivers/scsi/lpfc/lpfc_hw.h b/drivers/scsi/lpfc/lpfc_hw.h index 4083764916a5..634f8fff7425 100644 --- a/drivers/scsi/lpfc/lpfc_hw.h +++ b/drivers/scsi/lpfc/lpfc_hw.h @@ -608,6 +608,7 @@ struct fc_vft_header { #define ELS_CMD_LIRR 0x7A000000 #define ELS_CMD_LCB 0x81000000 #define ELS_CMD_FPIN 0x16000000 +#define ELS_CMD_EDC 0x17000000 #define ELS_CMD_QFPA 0xB0000000 #define ELS_CMD_UVEM 0xB1000000 #else /* __LITTLE_ENDIAN_BITFIELD */ @@ -652,6 +653,7 @@ struct fc_vft_header { #define ELS_CMD_LIRR 0x7A #define ELS_CMD_LCB 0x81 #define ELS_CMD_FPIN ELS_FPIN +#define ELS_CMD_EDC ELS_EDC #define ELS_CMD_QFPA 0xB0 #define ELS_CMD_UVEM 0xB1 #endif diff --git a/drivers/scsi/lpfc/lpfc_hw4.h b/drivers/scsi/lpfc/lpfc_hw4.h index 65bb4a66ccf0..ebee1d302a49 100644 --- a/drivers/scsi/lpfc/lpfc_hw4.h +++ b/drivers/scsi/lpfc/lpfc_hw4.h @@ -2813,6 +2813,12 @@ struct lpfc_mbx_read_config { #define lpfc_mbx_rd_conf_extnts_inuse_SHIFT 31 #define lpfc_mbx_rd_conf_extnts_inuse_MASK 0x00000001 #define lpfc_mbx_rd_conf_extnts_inuse_WORD word1 +#define lpfc_mbx_rd_conf_wcs_SHIFT 28 /* warning signaling */ +#define lpfc_mbx_rd_conf_wcs_MASK 0x00000001 +#define lpfc_mbx_rd_conf_wcs_WORD word1 +#define lpfc_mbx_rd_conf_acs_SHIFT 27 /* alarm signaling */ +#define lpfc_mbx_rd_conf_acs_MASK 0x00000001 +#define lpfc_mbx_rd_conf_acs_WORD word1 uint32_t word2; #define lpfc_mbx_rd_conf_lnk_numb_SHIFT 0 #define lpfc_mbx_rd_conf_lnk_numb_MASK 0x0000003F @@ -3393,6 +3399,7 @@ struct lpfc_sli4_parameters { #define LPFC_SET_UE_RECOVERY 0x10 #define LPFC_SET_MDS_DIAGS 0x12 +#define LPFC_SET_CGN_SIGNAL 0x1f #define LPFC_SET_DUAL_DUMP 0x1e #define LPFC_SET_ENABLE_MI 0x21 struct lpfc_mbx_set_feature { @@ -3409,6 +3416,9 @@ struct lpfc_mbx_set_feature { #define lpfc_mbx_set_feature_mds_deep_loopbk_SHIFT 1 #define lpfc_mbx_set_feature_mds_deep_loopbk_MASK 0x00000001 #define lpfc_mbx_set_feature_mds_deep_loopbk_WORD word6 +#define lpfc_mbx_set_feature_CGN_warn_freq_SHIFT 0 +#define lpfc_mbx_set_feature_CGN_warn_freq_MASK 0x0000ffff +#define lpfc_mbx_set_feature_CGN_warn_freq_WORD word6 #define lpfc_mbx_set_feature_dd_SHIFT 0 #define lpfc_mbx_set_feature_dd_MASK 0x00000001 #define lpfc_mbx_set_feature_dd_WORD word6 @@ -3431,6 +3441,13 @@ struct lpfc_mbx_set_feature { #define lpfc_mbx_set_feature_UESR_SHIFT 16 #define lpfc_mbx_set_feature_UESR_MASK 0x0000ffff #define lpfc_mbx_set_feature_UESR_WORD word7 +#define lpfc_mbx_set_feature_CGN_alarm_freq_SHIFT 0 +#define lpfc_mbx_set_feature_CGN_alarm_freq_MASK 0x0000ffff +#define lpfc_mbx_set_feature_CGN_alarm_freq_WORD word7 + u32 word8; +#define lpfc_mbx_set_feature_CGN_acqe_freq_SHIFT 0 +#define lpfc_mbx_set_feature_CGN_acqe_freq_MASK 0x000000ff +#define lpfc_mbx_set_feature_CGN_acqe_freq_WORD word8 }; @@ -4173,6 +4190,19 @@ struct lpfc_acqe_misconfigured_event { #define LPFC_SLI_EVENT_STATUS_UNCERTIFIED 0x05 }; +struct lpfc_acqe_cgn_signal { + u32 word0; +#define lpfc_warn_acqe_SHIFT 0 +#define lpfc_warn_acqe_MASK 0x7FFFFFFF +#define lpfc_warn_acqe_WORD word0 +#define lpfc_imm_acqe_SHIFT 31 +#define lpfc_imm_acqe_MASK 0x1 +#define lpfc_imm_acqe_WORD word0 + u32 alarm_cnt; + u32 word2; + u32 trailer; +}; + struct lpfc_acqe_sli { uint32_t event_data1; uint32_t event_data2; @@ -4187,6 +4217,7 @@ struct lpfc_acqe_sli { #define LPFC_SLI_EVENT_TYPE_REMOTE_DPORT 0xA #define LPFC_SLI_EVENT_TYPE_MISCONF_FAWWN 0xF #define LPFC_SLI_EVENT_TYPE_EEPROM_FAILURE 0x10 +#define LPFC_SLI_EVENT_TYPE_CGN_SIGNAL 0x11 }; /* @@ -4815,6 +4846,17 @@ struct lpfc_grp_hdr { #define LPFC_FW_RESET 2 #define LPFC_DV_RESET 3 +/* On some kernels, enum fc_ls_tlv_dtag does not have + * these 2 enums defined, on other kernels it does. + * To get aound this we need to add these 2 defines here. + */ +#ifndef ELS_DTAG_LNK_FAULT_CAP +#define ELS_DTAG_LNK_FAULT_CAP 0x0001000D +#endif +#ifndef ELS_DTAG_CG_SIGNAL_CAP +#define ELS_DTAG_CG_SIGNAL_CAP 0x0001000F +#endif + /* * Initializer useful for decoding FPIN string table. */ @@ -4823,6 +4865,22 @@ struct lpfc_grp_hdr { { FPIN_CONGN_SEVERITY_ERROR, "Alarm" }, \ } +/* EDC supports two descriptors. When allocated, it is the + * size of this structure plus each supported descriptor. + */ +struct lpfc_els_edc_req { + struct fc_els_edc edc; /* hdr up to descriptors */ + struct fc_diag_cg_sig_desc cgn_desc; /* 1st descriptor */ +}; + +/* Minimum structure defines for the EDC response. + * Balance is in buffer. + */ +struct lpfc_els_edc_rsp { + struct fc_els_edc_resp edc_rsp; /* hdr up to descriptors */ + struct fc_diag_cg_sig_desc cgn_desc; /* 1st descriptor */ +}; + /* Used for logging FPIN messages */ #define LPFC_FPIN_WWPN_LINE_SZ 128 #define LPFC_FPIN_WWPN_LINE_CNT 6 diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c index 6e75471525eb..9e4446302855 100644 --- a/drivers/scsi/lpfc/lpfc_init.c +++ b/drivers/scsi/lpfc/lpfc_init.c @@ -1243,7 +1243,8 @@ lpfc_idle_stat_delay_work(struct work_struct *work) return; if (phba->link_state == LPFC_HBA_ERROR || - phba->pport->fc_flag & FC_OFFLINE_MODE) + phba->pport->fc_flag & FC_OFFLINE_MODE || + phba->cmf_active_mode != LPFC_CFG_OFF) goto requeue; for_each_present_cpu(i) { @@ -5528,9 +5529,10 @@ lpfc_sli4_async_sli_evt(struct lpfc_hba *phba, struct lpfc_acqe_sli *acqe_sli) uint8_t operational = 0; struct temp_event temp_event_data; struct lpfc_acqe_misconfigured_event *misconfigured; + struct lpfc_acqe_cgn_signal *cgn_signal; struct Scsi_Host *shost; struct lpfc_vport **vports; - int rc, i; + int rc, i, cnt; evt_type = bf_get(lpfc_trailer_type, acqe_sli); @@ -5703,6 +5705,40 @@ lpfc_sli4_async_sli_evt(struct lpfc_hba *phba, struct lpfc_acqe_sli *acqe_sli) "Event Data1: x%08x Event Data2: x%08x\n", acqe_sli->event_data1, acqe_sli->event_data2); break; + case LPFC_SLI_EVENT_TYPE_CGN_SIGNAL: + if (phba->cmf_active_mode == LPFC_CFG_OFF) + break; + cgn_signal = (struct lpfc_acqe_cgn_signal *) + &acqe_sli->event_data1; + phba->cgn_acqe_cnt++; + + cnt = bf_get(lpfc_warn_acqe, cgn_signal); + atomic64_add(cnt, &phba->cgn_acqe_stat.warn); + atomic64_add(cgn_signal->alarm_cnt, &phba->cgn_acqe_stat.alarm); + + /* no threshold for CMF, even 1 signal will trigger an event */ + + /* Alarm overrides warning, so check that first */ + if (cgn_signal->alarm_cnt) { + if (phba->cgn_reg_signal == EDC_CG_SIG_WARN_ALARM) { + /* Keep track of alarm cnt for cgn_info */ + atomic_add(cgn_signal->alarm_cnt, + &phba->cgn_fabric_alarm_cnt); + /* Keep track of alarm cnt for CMF_SYNC_WQE */ + atomic_add(cgn_signal->alarm_cnt, + &phba->cgn_sync_alarm_cnt); + } + } else if (cnt) { + /* signal action needs to be taken */ + if (phba->cgn_reg_signal == EDC_CG_SIG_WARN_ONLY || + phba->cgn_reg_signal == EDC_CG_SIG_WARN_ALARM) { + /* Keep track of warning cnt for cgn_info */ + atomic_add(cnt, &phba->cgn_fabric_warn_cnt); + /* Keep track of warning cnt for CMF_SYNC_WQE */ + atomic_add(cnt, &phba->cgn_sync_warn_cnt); + } + } + break; default: lpfc_printf_log(phba, KERN_INFO, LOG_SLI, "3193 Unrecognized SLI event, type: 0x%x", @@ -8702,6 +8738,52 @@ lpfc_sli4_read_config(struct lpfc_hba *phba) phba->max_vpi = (phba->sli4_hba.max_cfg_param.max_vpi > 0) ? (phba->sli4_hba.max_cfg_param.max_vpi - 1) : 0; phba->max_vports = phba->max_vpi; + + /* Next decide on FPIN or Signal E2E CGN support + * For congestion alarms and warnings valid combination are: + * 1. FPIN alarms / FPIN warnings + * 2. Signal alarms / Signal warnings + * 3. FPIN alarms / Signal warnings + * 4. Signal alarms / FPIN warnings + * + * Initialize the adapter frequency to 100 mSecs + */ + phba->cgn_reg_fpin = LPFC_CGN_FPIN_BOTH; + phba->cgn_reg_signal = EDC_CG_SIG_NOTSUPPORTED; + phba->cgn_sig_freq = lpfc_fabric_cgn_frequency; + + if (lpfc_use_cgn_signal) { + if (bf_get(lpfc_mbx_rd_conf_wcs, rd_config)) { + phba->cgn_reg_signal = EDC_CG_SIG_WARN_ONLY; + phba->cgn_reg_fpin &= ~LPFC_CGN_FPIN_WARN; + } + if (bf_get(lpfc_mbx_rd_conf_acs, rd_config)) { + /* MUST support both alarm and warning + * because EDC does not support alarm alone. + */ + if (phba->cgn_reg_signal != + EDC_CG_SIG_WARN_ONLY) { + /* Must support both or none */ + phba->cgn_reg_fpin = LPFC_CGN_FPIN_BOTH; + phba->cgn_reg_signal = + EDC_CG_SIG_NOTSUPPORTED; + } else { + phba->cgn_reg_signal = + EDC_CG_SIG_WARN_ALARM; + phba->cgn_reg_fpin = + LPFC_CGN_FPIN_NONE; + } + } + } + + /* Set the congestion initial signal and fpin values. */ + phba->cgn_init_reg_fpin = phba->cgn_reg_fpin; + phba->cgn_init_reg_signal = phba->cgn_reg_signal; + + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "6446 READ_CONFIG reg_sig x%x reg_fpin:x%x\n", + phba->cgn_reg_signal, phba->cgn_reg_fpin); + lpfc_map_topology(phba, rd_config); lpfc_printf_log(phba, KERN_INFO, LOG_SLI, "2003 cfg params Extents? %d " diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index 5489cc7d06d5..3b6576d3be6d 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -6417,6 +6417,7 @@ lpfc_set_features(struct lpfc_hba *phba, LPFC_MBOXQ_t *mbox, uint32_t feature) { uint32_t len; + u32 sig_freq = 0; len = sizeof(struct lpfc_mbx_set_feature) - sizeof(struct lpfc_sli4_cfg_mhdr); @@ -6439,6 +6440,35 @@ lpfc_set_features(struct lpfc_hba *phba, LPFC_MBOXQ_t *mbox, mbox->u.mqe.un.set_feature.feature = LPFC_SET_MDS_DIAGS; mbox->u.mqe.un.set_feature.param_len = 8; break; + case LPFC_SET_CGN_SIGNAL: + if (phba->cmf_active_mode == LPFC_CFG_OFF) + sig_freq = 0; + else + sig_freq = phba->cgn_sig_freq; + + if (phba->cgn_reg_signal == EDC_CG_SIG_WARN_ALARM) { + bf_set(lpfc_mbx_set_feature_CGN_alarm_freq, + &mbox->u.mqe.un.set_feature, sig_freq); + bf_set(lpfc_mbx_set_feature_CGN_warn_freq, + &mbox->u.mqe.un.set_feature, sig_freq); + } + + if (phba->cgn_reg_signal == EDC_CG_SIG_WARN_ONLY) + bf_set(lpfc_mbx_set_feature_CGN_warn_freq, + &mbox->u.mqe.un.set_feature, sig_freq); + + if (phba->cmf_active_mode == LPFC_CFG_OFF || + phba->cgn_reg_signal == EDC_CG_SIG_NOTSUPPORTED) + sig_freq = 0; + else + sig_freq = lpfc_acqe_cgn_frequency; + + bf_set(lpfc_mbx_set_feature_CGN_acqe_freq, + &mbox->u.mqe.un.set_feature, sig_freq); + + mbox->u.mqe.un.set_feature.feature = LPFC_SET_CGN_SIGNAL; + mbox->u.mqe.un.set_feature.param_len = 12; + break; case LPFC_SET_DUAL_DUMP: bf_set(lpfc_mbx_set_feature_dd, &mbox->u.mqe.un.set_feature, LPFC_ENABLE_DUAL_DUMP); @@ -7445,6 +7475,91 @@ lpfc_post_rq_buffer(struct lpfc_hba *phba, struct lpfc_queue *hrq, return 1; } +static void +lpfc_mbx_cmpl_cgn_set_ftrs(struct lpfc_hba *phba, LPFC_MBOXQ_t *pmb) +{ + struct lpfc_vport *vport = pmb->vport; + union lpfc_sli4_cfg_shdr *shdr; + u32 shdr_status, shdr_add_status; + u32 sig, acqe; + + /* Two outcomes. (1) Set featurs was successul and EDC negotiation + * is done. (2) Mailbox failed and send FPIN support only. + */ + shdr = (union lpfc_sli4_cfg_shdr *) + &pmb->u.mqe.un.sli4_config.header.cfg_shdr; + shdr_status = bf_get(lpfc_mbox_hdr_status, &shdr->response); + shdr_add_status = bf_get(lpfc_mbox_hdr_add_status, &shdr->response); + if (shdr_status || shdr_add_status || pmb->u.mb.mbxStatus) { + lpfc_printf_log(phba, KERN_ERR, LOG_INIT | LOG_CGN_MGMT, + "2516 CGN SET_FEATURE mbox failed with " + "status x%x add_status x%x, mbx status x%x " + "Reset Congestion to FPINs only\n", + shdr_status, shdr_add_status, + pmb->u.mb.mbxStatus); + /* If there is a mbox error, move on to RDF */ + phba->cgn_reg_signal = EDC_CG_SIG_NOTSUPPORTED; + phba->cgn_reg_fpin = LPFC_CGN_FPIN_WARN | LPFC_CGN_FPIN_ALARM; + goto out; + } + + /* Zero out Congestion Signal ACQE counter */ + phba->cgn_acqe_cnt = 0; + atomic64_set(&phba->cgn_acqe_stat.warn, 0); + atomic64_set(&phba->cgn_acqe_stat.alarm, 0); + + acqe = bf_get(lpfc_mbx_set_feature_CGN_acqe_freq, + &pmb->u.mqe.un.set_feature); + sig = bf_get(lpfc_mbx_set_feature_CGN_warn_freq, + &pmb->u.mqe.un.set_feature); + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "4620 SET_FEATURES Success: Freq: %ds %dms " + " Reg: x%x x%x\n", acqe, sig, + phba->cgn_reg_signal, phba->cgn_reg_fpin); +out: + mempool_free(pmb, phba->mbox_mem_pool); + + /* Register for FPIN events from the fabric now that the + * EDC common_set_features has completed. + */ + lpfc_issue_els_rdf(vport, 0); +} + +int +lpfc_config_cgn_signal(struct lpfc_hba *phba) +{ + LPFC_MBOXQ_t *mboxq; + u32 rc; + + mboxq = (LPFC_MBOXQ_t *)mempool_alloc(phba->mbox_mem_pool, GFP_KERNEL); + if (!mboxq) + goto out_rdf; + + lpfc_set_features(phba, mboxq, LPFC_SET_CGN_SIGNAL); + mboxq->vport = phba->pport; + mboxq->mbox_cmpl = lpfc_mbx_cmpl_cgn_set_ftrs; + + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "4621 SET_FEATURES: FREQ sig x%x acqe x%x: " + "Reg: x%x x%x\n", + phba->cgn_sig_freq, lpfc_acqe_cgn_frequency, + phba->cgn_reg_signal, phba->cgn_reg_fpin); + + rc = lpfc_sli_issue_mbox(phba, mboxq, MBX_NOWAIT); + if (rc == MBX_NOT_FINISHED) + goto out; + return 0; + +out: + mempool_free(mboxq, phba->mbox_mem_pool); +out_rdf: + /* If there is a mbox error, move on to RDF */ + phba->cgn_reg_fpin = LPFC_CGN_FPIN_WARN | LPFC_CGN_FPIN_ALARM; + phba->cgn_reg_signal = EDC_CG_SIG_NOTSUPPORTED; + lpfc_issue_els_rdf(phba->pport, 0); + return -EIO; +} + /** * lpfc_init_idle_stat_hb - Initialize idle_stat tracking * @phba: pointer to lpfc hba data structure. @@ -7476,7 +7591,8 @@ static void lpfc_init_idle_stat_hb(struct lpfc_hba *phba) idle_stat->prev_idle = get_cpu_idle_time(i, &wall, 1); idle_stat->prev_wall = wall; - if (phba->nvmet_support) + if (phba->nvmet_support || + phba->cmf_active_mode != LPFC_CFG_OFF) cq->poll_mode = LPFC_QUEUE_WORK; else cq->poll_mode = LPFC_IRQ_POLL; @@ -9947,6 +10063,7 @@ lpfc_sli4_iocb2wqe(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq, if (pcmd && (*pcmd == ELS_CMD_FLOGI || *pcmd == ELS_CMD_SCR || *pcmd == ELS_CMD_RDF || + *pcmd == ELS_CMD_EDC || *pcmd == ELS_CMD_RSCN_XMT || *pcmd == ELS_CMD_FDISC || *pcmd == ELS_CMD_LOGO || @@ -14814,8 +14931,12 @@ static void lpfc_sli4_sched_cq_work(struct lpfc_hba *phba, switch (cq->poll_mode) { case LPFC_IRQ_POLL: - irq_poll_sched(&cq->iop); - break; + /* CGN mgmt is mutually exclusive from softirq processing */ + if (phba->cmf_active_mode == LPFC_CFG_OFF) { + irq_poll_sched(&cq->iop); + break; + } + fallthrough; case LPFC_QUEUE_WORK: default: if (is_kdump_kernel()) From patchwork Fri Aug 13 23:00:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 496858 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 869F7C4338F for ; Fri, 13 Aug 2021 23:01:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5E33D60FC4 for ; Fri, 13 Aug 2021 23:01:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235392AbhHMXBo (ORCPT ); Fri, 13 Aug 2021 19:01:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41578 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235440AbhHMXBg (ORCPT ); Fri, 13 Aug 2021 19:01:36 -0400 Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6D61BC0617AD for ; Fri, 13 Aug 2021 16:01:09 -0700 (PDT) Received: by mail-pj1-x102c.google.com with SMTP id a8so17504406pjk.4 for ; Fri, 13 Aug 2021 16:01:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=LkuYj2ysOwnCh8t89oLZM1e5OOYtT3bPSd8gwuIXLt8=; b=nUgYYWjccuM8DESSjp1uREOyX6Jw6f4O6FRaEbzGuP3O64LwELoVR+nGydPz9R5E7Y Nods61pQvjcBDP8tW/BedFsvIkbltqdWp0d+N9ak2cWYwHkK37DoZqSjnSXXaXpKlpWG oOM4pW9Jpi6EnWxICu+Z7RjMwjBdLCk5IogLRfQNpGgljqYJafYwtwnptHMv1Bzmsz6L cL1n795xjdMAeYhnnTEaPUvHXGqO95baY6dSV1yP/rgy3jGhYcZHZ3UnoXzNQlO7f8KK upD97bSRP60KQgpnsAZLBPRkM3DwMVvzk/EVHw4rx1hu1N3SUiDd3JlS69alZua9geOQ C5kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=LkuYj2ysOwnCh8t89oLZM1e5OOYtT3bPSd8gwuIXLt8=; b=JfDdSae5SWm8wF34hYUt8ch8gIH/LZ4JC+i+HFbdna3Tsfyz7J3l9L3+XZkCHCJBwc aEcoPMsIhfMBzGunVzDKZ0rmUJge9vYFMH32kvWFBlaat53I3hUVaIYqB1OZs7+289eA cSEjQzQreg0DoSX3LMw3CUiQXIe/mSMJr79PLk49M1nv3FKfwpgiCE8Kov0+YhcUwTLa fwtwJdc0/8PTxMrCIO+GZJ7lqkwxB8gPcU3j76R4sqmgZ4Cg0HQ/nN9ePAI2iibvbS7+ 3N9o3yhtLMWcPFF9ujRInrnSqoudHR673WQYXs5zg4ca5G0YKCGvBdWoYcvUTjX66ONa HJ2Q== X-Gm-Message-State: AOAM531cigjnvUD7UdkuwTeBv2SSmDhgV/xhve6prDKKS2ltYr77/p9C E/nwAOAfhEWfdXT9UiCPqX6JyodNVJ0= X-Google-Smtp-Source: ABdhPJxvxd2OcehDBf6XIA3CJGzl3d7/k9DxHnjaJmHKKYk2L5F9jiCWUhdGrgrhaNMA4BZkpw6C6w== X-Received: by 2002:a63:1952:: with SMTP id 18mr4300886pgz.15.1628895668859; Fri, 13 Aug 2021 16:01:08 -0700 (PDT) Received: from localhost.localdomain.localdomain ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id e8sm4001997pgg.31.2021.08.13.16.01.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 16:01:08 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH v2 08/16] lpfc: add cmfsync WQE support Date: Fri, 13 Aug 2021 16:00:31 -0700 Message-Id: <20210813230039.110546-9-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210813230039.110546-1-jsmart2021@gmail.com> References: <20210813230039.110546-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org When congestion mgmt is enabled, cmf has the driver regularly issue a command to synchronize reporting of congestion mgmt events such as fpin and signal delivery. This patch adds the definition of the CMF_SYNC WQE and its CQE fields as well as support for issuing the command. The patch also adds the few remaining cmf-related SLI additions, such as feature definition for enablement of CMF and notifications to the driver if the cm enablement mode changes. Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc.h | 4 + drivers/scsi/lpfc/lpfc_crtn.h | 1 + drivers/scsi/lpfc/lpfc_hw4.h | 87 ++++++++++++++++- drivers/scsi/lpfc/lpfc_sli.c | 178 ++++++++++++++++++++++++++++++++++ drivers/scsi/lpfc/lpfc_sli.h | 1 + 5 files changed, 268 insertions(+), 3 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h index 5a356a1d517c..12972bfed923 100644 --- a/drivers/scsi/lpfc/lpfc.h +++ b/drivers/scsi/lpfc/lpfc.h @@ -1499,6 +1499,10 @@ struct lpfc_hba { u32 cmf_active_mode; #define LPFC_CFG_OFF 0 +#define LPFC_CMF_INTERVAL 90 +#define LPFC_CMF_BLK_SIZE 512 +#define LPFC_MAX_CMF_INFO 32 + /* Signal / FPIN handling for Congestion Mgmt */ u8 cgn_reg_fpin; /* Negotiated value from RDF */ u8 cgn_init_reg_fpin; /* Initial value from READ_CONFIG */ diff --git a/drivers/scsi/lpfc/lpfc_crtn.h b/drivers/scsi/lpfc/lpfc_crtn.h index 947c4ba847f6..3621acf9437d 100644 --- a/drivers/scsi/lpfc/lpfc_crtn.h +++ b/drivers/scsi/lpfc/lpfc_crtn.h @@ -79,6 +79,7 @@ void lpfc_init_congestion_stat(struct lpfc_hba *phba); void lpfc_init_congestion_buf(struct lpfc_hba *phba); int lpfc_sli4_cgn_params_read(struct lpfc_hba *phba); int lpfc_config_cgn_signal(struct lpfc_hba *phba); +int lpfc_issue_cmf_sync_wqe(struct lpfc_hba *phba, u32 ms, u64 total); void lpfc_mbx_cmpl_local_config_link(struct lpfc_hba *, LPFC_MBOXQ_t *); void lpfc_mbx_cmpl_reg_login(struct lpfc_hba *, LPFC_MBOXQ_t *); diff --git a/drivers/scsi/lpfc/lpfc_hw4.h b/drivers/scsi/lpfc/lpfc_hw4.h index 973af1f86d28..73b249d0d964 100644 --- a/drivers/scsi/lpfc/lpfc_hw4.h +++ b/drivers/scsi/lpfc/lpfc_hw4.h @@ -397,6 +397,12 @@ struct lpfc_wcqe_complete { #define lpfc_wcqe_c_ersp0_MASK 0x0000FFFF #define lpfc_wcqe_c_ersp0_WORD word0 uint32_t total_data_placed; +#define lpfc_wcqe_c_cmf_cg_SHIFT 31 +#define lpfc_wcqe_c_cmf_cg_MASK 0x00000001 +#define lpfc_wcqe_c_cmf_cg_WORD total_data_placed +#define lpfc_wcqe_c_cmf_bw_SHIFT 0 +#define lpfc_wcqe_c_cmf_bw_MASK 0x0FFFFFFF +#define lpfc_wcqe_c_cmf_bw_WORD total_data_placed uint32_t parameter; #define lpfc_wcqe_c_bg_edir_SHIFT 5 #define lpfc_wcqe_c_bg_edir_MASK 0x00000001 @@ -691,6 +697,7 @@ struct lpfc_register { #define lpfc_sliport_eqdelay_id_MASK 0xfff #define lpfc_sliport_eqdelay_id_WORD word0 #define LPFC_SEC_TO_USEC 1000000 +#define LPFC_SEC_TO_MSEC 1000 /* The following Registers apply to SLI4 if_type 0 UCNAs. They typically * reside in BAR 2. @@ -3397,12 +3404,13 @@ struct lpfc_sli4_parameters { #define cfg_max_tow_xri_WORD word20 uint32_t word21; -#define cfg_mib_bde_cnt_SHIFT 16 -#define cfg_mib_bde_cnt_MASK 0x000000ff -#define cfg_mib_bde_cnt_WORD word21 #define cfg_mi_ver_SHIFT 0 #define cfg_mi_ver_MASK 0x0000ffff #define cfg_mi_ver_WORD word21 +#define cfg_cmf_SHIFT 24 +#define cfg_cmf_MASK 0x000000ff +#define cfg_cmf_WORD word21 + uint32_t mib_size; uint32_t word23; /* RESERVED */ @@ -3434,6 +3442,7 @@ struct lpfc_sli4_parameters { #define LPFC_SET_CGN_SIGNAL 0x1f #define LPFC_SET_DUAL_DUMP 0x1e #define LPFC_SET_ENABLE_MI 0x21 +#define LPFC_SET_ENABLE_CMF 0x24 struct lpfc_mbx_set_feature { struct mbox_header header; uint32_t feature; @@ -3460,6 +3469,9 @@ struct lpfc_mbx_set_feature { #define LPFC_DISABLE_DUAL_DUMP 0 #define LPFC_ENABLE_DUAL_DUMP 1 #define LPFC_QUERY_OP_DUAL_DUMP 2 +#define lpfc_mbx_set_feature_cmf_SHIFT 0 +#define lpfc_mbx_set_feature_cmf_MASK 0x00000001 +#define lpfc_mbx_set_feature_cmf_WORD word6 #define lpfc_mbx_set_feature_mi_SHIFT 0 #define lpfc_mbx_set_feature_mi_MASK 0x0000ffff #define lpfc_mbx_set_feature_mi_WORD word6 @@ -4005,6 +4017,7 @@ struct lpfc_mcqe { #define LPFC_TRAILER_CODE_GRP5 0x5 #define LPFC_TRAILER_CODE_FC 0x10 #define LPFC_TRAILER_CODE_SLI 0x11 +#define LPFC_TRAILER_CODE_CMSTAT 0x13 }; struct lpfc_acqe_link { @@ -4264,6 +4277,7 @@ struct lpfc_acqe_sli { #define LPFC_SLI_EVENT_TYPE_DIAG_DUMP 0x5 #define LPFC_SLI_EVENT_TYPE_MISCONFIGURED 0x9 #define LPFC_SLI_EVENT_TYPE_REMOTE_DPORT 0xA +#define LPFC_SLI_EVENT_TYPE_PORT_PARAMS_CHG 0xE #define LPFC_SLI_EVENT_TYPE_MISCONF_FAWWN 0xF #define LPFC_SLI_EVENT_TYPE_EEPROM_FAILURE 0x10 #define LPFC_SLI_EVENT_TYPE_CGN_SIGNAL 0x11 @@ -4674,6 +4688,69 @@ struct create_xri_wqe { #define T_REQUEST_TAG 3 #define T_XRI_TAG 1 +struct cmf_sync_wqe { + uint32_t rsrvd[3]; + uint32_t word3; +#define cmf_sync_interval_SHIFT 0 +#define cmf_sync_interval_MASK 0x00000ffff +#define cmf_sync_interval_WORD word3 +#define cmf_sync_afpin_SHIFT 16 +#define cmf_sync_afpin_MASK 0x000000001 +#define cmf_sync_afpin_WORD word3 +#define cmf_sync_asig_SHIFT 17 +#define cmf_sync_asig_MASK 0x000000001 +#define cmf_sync_asig_WORD word3 +#define cmf_sync_op_SHIFT 20 +#define cmf_sync_op_MASK 0x00000000f +#define cmf_sync_op_WORD word3 +#define cmf_sync_ver_SHIFT 24 +#define cmf_sync_ver_MASK 0x0000000ff +#define cmf_sync_ver_WORD word3 +#define LPFC_CMF_SYNC_VER 1 + uint32_t event_tag; + uint32_t word5; +#define cmf_sync_wsigmax_SHIFT 0 +#define cmf_sync_wsigmax_MASK 0x00000ffff +#define cmf_sync_wsigmax_WORD word5 +#define cmf_sync_wsigcnt_SHIFT 16 +#define cmf_sync_wsigcnt_MASK 0x00000ffff +#define cmf_sync_wsigcnt_WORD word5 + uint32_t word6; + uint32_t word7; +#define cmf_sync_cmnd_SHIFT 8 +#define cmf_sync_cmnd_MASK 0x0000000ff +#define cmf_sync_cmnd_WORD word7 + uint32_t word8; + uint32_t word9; +#define cmf_sync_reqtag_SHIFT 0 +#define cmf_sync_reqtag_MASK 0x00000ffff +#define cmf_sync_reqtag_WORD word9 +#define cmf_sync_wfpinmax_SHIFT 16 +#define cmf_sync_wfpinmax_MASK 0x0000000ff +#define cmf_sync_wfpinmax_WORD word9 +#define cmf_sync_wfpincnt_SHIFT 24 +#define cmf_sync_wfpincnt_MASK 0x0000000ff +#define cmf_sync_wfpincnt_WORD word9 + uint32_t word10; +#define cmf_sync_qosd_SHIFT 9 +#define cmf_sync_qosd_MASK 0x00000001 +#define cmf_sync_qosd_WORD word10 + uint32_t word11; +#define cmf_sync_cmd_type_SHIFT 0 +#define cmf_sync_cmd_type_MASK 0x0000000f +#define cmf_sync_cmd_type_WORD word11 +#define cmf_sync_wqec_SHIFT 7 +#define cmf_sync_wqec_MASK 0x00000001 +#define cmf_sync_wqec_WORD word11 +#define cmf_sync_cqid_SHIFT 16 +#define cmf_sync_cqid_MASK 0x0000ffff +#define cmf_sync_cqid_WORD word11 + uint32_t read_bytes; + uint32_t word13; + uint32_t word14; + uint32_t word15; +}; + struct abort_cmd_wqe { uint32_t rsrvd[3]; uint32_t word3; @@ -4803,6 +4880,7 @@ union lpfc_wqe { struct fcp_iread64_wqe fcp_iread; struct fcp_iwrite64_wqe fcp_iwrite; struct abort_cmd_wqe abort_cmd; + struct cmf_sync_wqe cmf_sync; struct create_xri_wqe create_xri; struct xmit_bcast64_wqe xmit_bcast64; struct xmit_seq64_wqe xmit_sequence; @@ -4823,6 +4901,7 @@ union lpfc_wqe128 { struct fcp_iread64_wqe fcp_iread; struct fcp_iwrite64_wqe fcp_iwrite; struct abort_cmd_wqe abort_cmd; + struct cmf_sync_wqe cmf_sync; struct create_xri_wqe create_xri; struct xmit_bcast64_wqe xmit_bcast64; struct xmit_seq64_wqe xmit_sequence; @@ -4866,6 +4945,7 @@ struct lpfc_grp_hdr { #define FCP_COMMAND_TRSP 0x3 #define FCP_COMMAND_TSEND 0x7 #define OTHER_COMMAND 0x8 +#define CMF_SYNC_COMMAND 0xA #define ELS_COMMAND_NON_FIP 0xC #define ELS_COMMAND_FIP 0xD @@ -4887,6 +4967,7 @@ struct lpfc_grp_hdr { #define CMD_FCP_TRECEIVE64_WQE 0xA1 #define CMD_FCP_TRSP64_WQE 0xA3 #define CMD_GEN_REQUEST64_WQE 0xC2 +#define CMD_CMF_SYNC_WQE 0xE8 #define CMD_WQE_MASK 0xff diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index b42c2dc49c83..4d1c190823d1 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -1768,6 +1768,184 @@ lpfc_sli_ringtx_get(struct lpfc_hba *phba, struct lpfc_sli_ring *pring) return cmd_iocb; } +/** + * lpfc_cmf_sync_cmpl - Process a CMF_SYNC_WQE cmpl + * @phba: Pointer to HBA context object. + * @cmdiocb: Pointer to driver command iocb object. + * @cmf_cmpl: Pointer to completed WCQE. + * + * This routine will inform the driver of any BW adjustments we need + * to make. These changes will be picked up during the next CMF + * timer interrupt. In addition, any BW changes will be logged + * with LOG_CGN_MGMT. + **/ +static void +lpfc_cmf_sync_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, + struct lpfc_wcqe_complete *cmf_cmpl) +{ + union lpfc_wqe128 *wqe; + uint32_t status, info; + uint64_t bw; + int asig, afpin, sigcnt, fpincnt; + int cg, tdp; + + /* First check for error */ + status = bf_get(lpfc_wcqe_c_status, cmf_cmpl); + if (status) { + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "6211 CMF_SYNC_WQE Error " + "req_tag x%x status x%x hwstatus x%x " + "tdatap x%x parm x%x\n", + bf_get(lpfc_wcqe_c_request_tag, cmf_cmpl), + bf_get(lpfc_wcqe_c_status, cmf_cmpl), + bf_get(lpfc_wcqe_c_hw_status, cmf_cmpl), + cmf_cmpl->total_data_placed, + cmf_cmpl->parameter); + goto out; + } + + /* Gather congestion information on a successful cmpl */ + info = cmf_cmpl->parameter; + tdp = bf_get(lpfc_wcqe_c_cmf_bw, cmf_cmpl); + cg = bf_get(lpfc_wcqe_c_cmf_cg, cmf_cmpl); + + /* Get BW requirement from firmware */ + bw = (uint64_t)tdp * LPFC_CMF_BLK_SIZE; + if (!bw) { + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "6212 CMF_SYNC_WQE x%x: NULL bw\n", + bf_get(lpfc_wcqe_c_request_tag, cmf_cmpl)); + goto out; + } + + /* Gather information needed for logging if a BW change is required */ + wqe = &cmdiocb->wqe; + asig = bf_get(cmf_sync_asig, &wqe->cmf_sync); + afpin = bf_get(cmf_sync_afpin, &wqe->cmf_sync); + fpincnt = bf_get(cmf_sync_wfpincnt, &wqe->cmf_sync); + sigcnt = bf_get(cmf_sync_wsigcnt, &wqe->cmf_sync); + +out: + lpfc_sli_release_iocbq(phba, cmdiocb); +} + +/** + * lpfc_issue_cmf_sync_wqe - Issue a CMF_SYNC_WQE + * @phba: Pointer to HBA context object. + * @ms: ms to set in WQE interval, 0 means use init op + * @total: Total rcv bytes for this interval + * + * This routine is called every CMF timer interrupt. Its purpose is + * to issue a CMF_SYNC_WQE to the firmware to inform it of any events + * that may indicate we have congestion (FPINs or Signals). Upon + * completion, the firmware will indicate any BW restrictions the + * driver may need to take. + **/ +int +lpfc_issue_cmf_sync_wqe(struct lpfc_hba *phba, u32 ms, u64 total) +{ + union lpfc_wqe128 *wqe; + struct lpfc_iocbq *sync_buf; + unsigned long iflags; + u32 ret_val; + u32 atot, wtot, max; + + /* First address any alarm / warning activity */ + atot = atomic_xchg(&phba->cgn_sync_alarm_cnt, 0); + wtot = atomic_xchg(&phba->cgn_sync_warn_cnt, 0); + + /* ONLY Managed mode will send the CMF_SYNC_WQE to the HBA */ + if (phba->cmf_active_mode != LPFC_CFG_MANAGED || + phba->link_state == LPFC_LINK_DOWN) + return 0; + + spin_lock_irqsave(&phba->hbalock, iflags); + sync_buf = __lpfc_sli_get_iocbq(phba); + if (!sync_buf) { + lpfc_printf_log(phba, KERN_ERR, LOG_CGN_MGMT, + "6213 No available WQEs for CMF_SYNC_WQE\n"); + ret_val = ENOMEM; + goto out_unlock; + } + + wqe = &sync_buf->wqe; + + /* WQEs are reused. Clear stale data and set key fields to zero */ + memset(wqe, 0, sizeof(*wqe)); + + /* If this is the very first CMF_SYNC_WQE, issue an init operation */ + if (!ms) { + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "6441 CMF Init %d - CMF_SYNC_WQE\n", + phba->fc_eventTag); + bf_set(cmf_sync_op, &wqe->cmf_sync, 1); /* 1=init */ + bf_set(cmf_sync_interval, &wqe->cmf_sync, LPFC_CMF_INTERVAL); + goto initpath; + } + + bf_set(cmf_sync_op, &wqe->cmf_sync, 0); /* 0=recalc */ + bf_set(cmf_sync_interval, &wqe->cmf_sync, ms); + + /* Check for alarms / warnings */ + if (atot) { + if (phba->cgn_reg_signal == EDC_CG_SIG_WARN_ALARM) { + /* We hit an Signal alarm condition */ + bf_set(cmf_sync_asig, &wqe->cmf_sync, 1); + } else { + /* We hit a FPIN alarm condition */ + bf_set(cmf_sync_afpin, &wqe->cmf_sync, 1); + } + } else if (wtot) { + if (phba->cgn_reg_signal == EDC_CG_SIG_WARN_ONLY || + phba->cgn_reg_signal == EDC_CG_SIG_WARN_ALARM) { + /* We hit an Signal warning condition */ + max = LPFC_SEC_TO_MSEC / lpfc_fabric_cgn_frequency * + lpfc_acqe_cgn_frequency; + bf_set(cmf_sync_wsigmax, &wqe->cmf_sync, max); + bf_set(cmf_sync_wsigcnt, &wqe->cmf_sync, wtot); + } else { + /* We hit a FPIN warning condition */ + bf_set(cmf_sync_wfpinmax, &wqe->cmf_sync, 1); + bf_set(cmf_sync_wfpincnt, &wqe->cmf_sync, 1); + } + } + + /* Update total read blocks during previous timer interval */ + wqe->cmf_sync.read_bytes = (u32)(total / LPFC_CMF_BLK_SIZE); + +initpath: + bf_set(cmf_sync_ver, &wqe->cmf_sync, LPFC_CMF_SYNC_VER); + wqe->cmf_sync.event_tag = phba->fc_eventTag; + bf_set(cmf_sync_cmnd, &wqe->cmf_sync, CMD_CMF_SYNC_WQE); + + /* Setup reqtag to match the wqe completion. */ + bf_set(cmf_sync_reqtag, &wqe->cmf_sync, sync_buf->iotag); + + bf_set(cmf_sync_qosd, &wqe->cmf_sync, 1); + + bf_set(cmf_sync_cmd_type, &wqe->cmf_sync, CMF_SYNC_COMMAND); + bf_set(cmf_sync_wqec, &wqe->cmf_sync, 1); + bf_set(cmf_sync_cqid, &wqe->cmf_sync, LPFC_WQE_CQ_ID_DEFAULT); + + sync_buf->vport = phba->pport; + sync_buf->wqe_cmpl = lpfc_cmf_sync_cmpl; + sync_buf->iocb_cmpl = NULL; + sync_buf->context1 = NULL; + sync_buf->context2 = NULL; + sync_buf->context3 = NULL; + sync_buf->sli4_xritag = NO_XRI; + + sync_buf->iocb_flag |= LPFC_IO_CMF; + ret_val = lpfc_sli4_issue_wqe(phba, &phba->sli4_hba.hdwq[0], sync_buf); + if (ret_val) + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "6214 Cannot issue CMF_SYNC_WQE: x%x\n", + ret_val); +out_unlock: + spin_unlock_irqrestore(&phba->hbalock, iflags); + return ret_val; +} + /** * lpfc_sli_next_iocb_slot - Get next iocb slot in the ring * @phba: Pointer to HBA context object. diff --git a/drivers/scsi/lpfc/lpfc_sli.h b/drivers/scsi/lpfc/lpfc_sli.h index dde8eb9d796d..dc7cc2f37089 100644 --- a/drivers/scsi/lpfc/lpfc_sli.h +++ b/drivers/scsi/lpfc/lpfc_sli.h @@ -107,6 +107,7 @@ struct lpfc_iocbq { #define LPFC_IO_NVME_LS 0x400000 /* NVME LS command */ #define LPFC_IO_NVMET 0x800000 /* NVMET command */ #define LPFC_IO_VMID 0x1000000 /* VMID tagged IO */ +#define LPFC_IO_CMF 0x4000000 /* CMF command */ uint32_t drvrTimeout; /* driver timeout in seconds */ struct lpfc_vport *vport;/* virtual port pointer */ From patchwork Fri Aug 13 23:00:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 496857 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 202A4C432BE for ; Fri, 13 Aug 2021 23:01:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 00FFE60724 for ; Fri, 13 Aug 2021 23:01:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235442AbhHMXBq (ORCPT ); Fri, 13 Aug 2021 19:01:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41588 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235469AbhHMXBj (ORCPT ); Fri, 13 Aug 2021 19:01:39 -0400 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3C940C061756 for ; Fri, 13 Aug 2021 16:01:12 -0700 (PDT) Received: by mail-pj1-x102d.google.com with SMTP id oa17so17541076pjb.1 for ; Fri, 13 Aug 2021 16:01:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qUXA5md1rXWOeLqiYSkY0N9/kg0WrnoYZAkhzLWBERA=; b=J2WUGdaoJssExwB65d95/KZeGnzvN9PQpbfFv7oEO2WkM6zJT65DsPqvbMStICtHUk 1zUGNIXDR5wzhaHvzAIlwok2rbpNhFzcyr3UoXjTLgZz7C6shwJBAUWVBTyAtPYgAx5S t3R1vQL1oy049s1wcL+fk2CCsrrdVhfEe/0xszMG7/bdE21JqghLUZwDpyITSoT5nb2o /XHLKZ972lPFTTntM3j0iUaL1np1BsE0LXI4GDRxZ3BdZfUe/e3M/DiryaRSGobpbjfr Ut951XQvOykiEv5jxfCE7NlRcpzwTxUH1mmsXkh6LZEjg8LwYu4JvLRYHjrdyifyIVAf ifZg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qUXA5md1rXWOeLqiYSkY0N9/kg0WrnoYZAkhzLWBERA=; b=PXriyLn3p5B2AcnfgtJdrvGlbOlxTJE1W4hLJPYbEIGTaY6ZWRRWoWcaJwkyEjF7XU Zfr0F7IDyI6gfWPQB/eM5eBOzNrxEzBAAf/abfHtHlWjid1z0WDP8TcNS0rzqCjAwsuj GCWJ7s8zT71KK6mi2n5qAhwhTe3mGBrrexBIpBR4IQetxbMYEPMB/w7hKJ0ZyBN99Ys8 qkWBE+3wc+FA/v6TyrAEU69WWhYh/1Qr8TfHrvUZ3xTwFtVZ37uanzhg9vvkpzF0Gavn M+e0xa4DWidmqj5N5yVIcN6WF2JryNlQIHb9CpES/tZGPnLi6yvdEIvM+y1OeYYkh2qN rpFw== X-Gm-Message-State: AOAM530/pmJT4eYdjwycRfKWfqF5YpRfo1UGUM1Gm0N17BI/1KSFl0TY 7wV7+UEx4tFFZGoXC4yFOiCT9xpE6ao= X-Google-Smtp-Source: ABdhPJy7jYgrYAf/9jwhGf0cM6s8tP1PRuS7v0A6fe8IKQWZdkPioT+7ntvsUKCRYPzMdGPg2XXwqA== X-Received: by 2002:a63:7405:: with SMTP id p5mr4281630pgc.426.1628895671350; Fri, 13 Aug 2021 16:01:11 -0700 (PDT) Received: from localhost.localdomain.localdomain ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id e8sm4001997pgg.31.2021.08.13.16.01.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 16:01:11 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH v2 09/16] lpfc: Add support for the CM framework Date: Fri, 13 Aug 2021 16:00:32 -0700 Message-Id: <20210813230039.110546-10-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210813230039.110546-1-jsmart2021@gmail.com> References: <20210813230039.110546-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org This patch completes the enablement of the cm framework feature in the adapter. The patch performs the following: - Detects the presence of the congestion management framework feature When the cm framework is present: - Issues the SET_FEATURE command to enable the feature - Registers the cm statistics buffer with the adapter - Reads the cm enablement buffer to determine the cm framework state for cm management. When cm management is enabled: - Monitors all FPIN and congestion signalling events, incrementing counters. - Regularly syncs with the adapter to communicate congestion events and to receive an rx request limit. - Monitors requests for rx data and ensures that no more than the adapter prescribed limit is issued on the link. If the limit is exceeded, scsi and/or nvme traffic is temporarily suspended. - Maintains the minute, hourly, daily statistics buffer. - Monitors for congestion enablement change events, causing a reread of the enablement buffer and acting on any change in enablement. And - Adds teardown logic, including buffer deregistration, on adapter detachment or reset. Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart Reported-by: kernel test robot --- drivers/scsi/lpfc/lpfc.h | 33 ++- drivers/scsi/lpfc/lpfc_attr.c | 1 + drivers/scsi/lpfc/lpfc_crtn.h | 10 + drivers/scsi/lpfc/lpfc_els.c | 73 +++++- drivers/scsi/lpfc/lpfc_hbadisc.c | 4 + drivers/scsi/lpfc/lpfc_init.c | 391 ++++++++++++++++++++++++++++++- drivers/scsi/lpfc/lpfc_mem.c | 9 + drivers/scsi/lpfc/lpfc_nvme.c | 44 +++- drivers/scsi/lpfc/lpfc_scsi.c | 185 ++++++++++++++- drivers/scsi/lpfc/lpfc_sli.c | 192 ++++++++++++++- drivers/scsi/lpfc/lpfc_sli.h | 1 + drivers/scsi/lpfc/lpfc_sli4.h | 1 + 12 files changed, 910 insertions(+), 34 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h index 12972bfed923..298b908e9126 100644 --- a/drivers/scsi/lpfc/lpfc.h +++ b/drivers/scsi/lpfc/lpfc.h @@ -550,6 +550,14 @@ struct lpfc_cgn_info { #define LPFC_CGN_INFO_SZ (sizeof(struct lpfc_cgn_info) - \ sizeof(uint32_t)) +struct lpfc_cgn_stat { + atomic64_t total_bytes; + atomic64_t rcv_bytes; + atomic64_t rx_latency; +#define LPFC_CGN_NOT_SENT 0xFFFFFFFFFFFFFFFFLL + atomic_t rx_io_cnt; +}; + struct lpfc_cgn_acqe_stat { atomic64_t alarm; atomic64_t warn; @@ -1021,7 +1029,10 @@ struct lpfc_hba { * capability */ #define HBA_FLOGI_ISSUED 0x100000 /* FLOGI was issued */ +#define HBA_CGN_RSVD1 0x200000 /* Reserved CGN flag */ +#define HBA_CGN_DAY_WRAP 0x400000 /* HBA Congestion info day wraps */ #define HBA_DEFER_FLOGI 0x800000 /* Defer FLOGI till read_sparm cmpl */ +#define HBA_SETUP 0x1000000 /* Signifies HBA setup is completed */ #define HBA_NEEDS_CFG_PORT 0x2000000 /* SLI3 - needs a CONFIG_PORT mbox */ #define HBA_HBEAT_INP 0x4000000 /* mbox HBEAT is in progress */ #define HBA_HBEAT_TMO 0x8000000 /* HBEAT initiated after timeout */ @@ -1272,6 +1283,7 @@ struct lpfc_hba { uint32_t total_iocbq_bufs; struct list_head active_rrq_list; spinlock_t hbalock; + struct work_struct unblock_request_work; /* SCSI layer unblock IOs */ /* dma_mem_pools */ struct dma_pool *lpfc_sg_dma_buf_pool; @@ -1496,12 +1508,25 @@ struct lpfc_hba { uint64_t ktime_seg10_max; #endif /* CMF objects */ - u32 cmf_active_mode; -#define LPFC_CFG_OFF 0 - + struct lpfc_cgn_stat __percpu *cmf_stat; + uint32_t cmf_interval_rate; /* timer interval limit in ms */ + uint32_t cmf_timer_cnt; #define LPFC_CMF_INTERVAL 90 + uint64_t cmf_link_byte_count; + uint64_t cmf_max_line_rate; + uint64_t cmf_max_bytes_per_interval; + uint64_t cmf_last_sync_bw; #define LPFC_CMF_BLK_SIZE 512 + struct hrtimer cmf_timer; + atomic_t cmf_bw_wait; + atomic_t cmf_busy; + atomic_t cmf_stop_io; /* To block request and stop IO's */ + uint32_t cmf_active_mode; + uint32_t cmf_info_per_interval; #define LPFC_MAX_CMF_INFO 32 + struct timespec64 cmf_latency; /* Interval congestion timestamp */ + uint32_t cmf_last_ts; /* Interval congestion time (ms) */ + uint32_t cmf_active_info; /* Signal / FPIN handling for Congestion Mgmt */ u8 cgn_reg_fpin; /* Negotiated value from RDF */ @@ -1521,6 +1546,8 @@ struct lpfc_hba { u32 cgn_sig_freq; u32 cgn_acqe_cnt; + uint64_t rx_block_cnt; + /* Congestion parameters from flash */ struct lpfc_cgn_param cgn_p; diff --git a/drivers/scsi/lpfc/lpfc_attr.c b/drivers/scsi/lpfc/lpfc_attr.c index 449409cad60d..b41891aefa64 100644 --- a/drivers/scsi/lpfc/lpfc_attr.c +++ b/drivers/scsi/lpfc/lpfc_attr.c @@ -7476,6 +7476,7 @@ lpfc_get_cfgparam(struct lpfc_hba *phba) lpfc_enable_dpp_init(phba, lpfc_enable_dpp); lpfc_enable_mi_init(phba, lpfc_enable_mi); + phba->cgn_p.cgn_param_mode = LPFC_CFG_OFF; phba->cmf_active_mode = LPFC_CFG_OFF; if (lpfc_fabric_cgn_frequency > EDC_CG_SIGFREQ_CNT_MAX || lpfc_fabric_cgn_frequency < EDC_CG_SIGFREQ_CNT_MIN) diff --git a/drivers/scsi/lpfc/lpfc_crtn.h b/drivers/scsi/lpfc/lpfc_crtn.h index 3621acf9437d..3addb163c2cd 100644 --- a/drivers/scsi/lpfc/lpfc_crtn.h +++ b/drivers/scsi/lpfc/lpfc_crtn.h @@ -59,6 +59,7 @@ int lpfc_sli4_mbox_rsrc_extent(struct lpfc_hba *, struct lpfcMboxq *, uint16_t, uint16_t, bool); int lpfc_get_sli4_parameters(struct lpfc_hba *, LPFC_MBOXQ_t *); int lpfc_reg_congestion_buf(struct lpfc_hba *phba); +int lpfc_unreg_congestion_buf(struct lpfc_hba *phba); struct lpfc_vport *lpfc_find_vport_by_did(struct lpfc_hba *, uint32_t); void lpfc_cleanup_rcv_buffers(struct lpfc_vport *); void lpfc_rcv_seq_check_edtov(struct lpfc_vport *); @@ -75,11 +76,17 @@ int lpfc_init_iocb_list(struct lpfc_hba *phba, int cnt); void lpfc_free_iocb_list(struct lpfc_hba *phba); int lpfc_post_rq_buffer(struct lpfc_hba *phba, struct lpfc_queue *hrq, struct lpfc_queue *drq, int count, int idx); +uint32_t lpfc_calc_cmf_latency(struct lpfc_hba *phba); +void lpfc_cmf_signal_init(struct lpfc_hba *phba); +void lpfc_cmf_start(struct lpfc_hba *phba); +void lpfc_cmf_stop(struct lpfc_hba *phba); void lpfc_init_congestion_stat(struct lpfc_hba *phba); void lpfc_init_congestion_buf(struct lpfc_hba *phba); int lpfc_sli4_cgn_params_read(struct lpfc_hba *phba); int lpfc_config_cgn_signal(struct lpfc_hba *phba); int lpfc_issue_cmf_sync_wqe(struct lpfc_hba *phba, u32 ms, u64 total); +void lpfc_unblock_requests(struct lpfc_hba *phba); +void lpfc_block_requests(struct lpfc_hba *phba); void lpfc_mbx_cmpl_local_config_link(struct lpfc_hba *, LPFC_MBOXQ_t *); void lpfc_mbx_cmpl_reg_login(struct lpfc_hba *, LPFC_MBOXQ_t *); @@ -471,6 +478,9 @@ void lpfc_free_fast_evt(struct lpfc_hba *, struct lpfc_fast_path_event *); void lpfc_create_static_vport(struct lpfc_hba *); void lpfc_stop_hba_timers(struct lpfc_hba *); void lpfc_stop_port(struct lpfc_hba *); +int lpfc_update_cmf_cmd(struct lpfc_hba *phba, uint32_t sz); +int lpfc_update_cmf_cmpl(struct lpfc_hba *phba, uint64_t val, uint32_t sz, + struct Scsi_Host *shost); void __lpfc_sli4_stop_fcf_redisc_wait_timer(struct lpfc_hba *); void lpfc_sli4_stop_fcf_redisc_wait_timer(struct lpfc_hba *); void lpfc_parse_fcoe_conf(struct lpfc_hba *, uint8_t *, uint32_t); diff --git a/drivers/scsi/lpfc/lpfc_els.c b/drivers/scsi/lpfc/lpfc_els.c index 3b1b0d3ff0d9..d6e64a6c5c07 100644 --- a/drivers/scsi/lpfc/lpfc_els.c +++ b/drivers/scsi/lpfc/lpfc_els.c @@ -3333,9 +3333,11 @@ lpfc_cmpl_els_disc_cmd(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS | LOG_CGN_MGMT, "4677 Fabric RDF Notification Grant " - "Data: 0x%08x\n", + "Data: 0x%08x Reg: %x %x\n", be32_to_cpu( - prdf->reg_d1.desc_tags[i])); + prdf->reg_d1.desc_tags[i]), + phba->cgn_reg_signal, + phba->cgn_reg_fpin); } out: @@ -3702,9 +3704,11 @@ lpfc_issue_els_rdf(struct lpfc_vport *vport, uint8_t retry) prdf->reg_d1.desc_tags[3] = cpu_to_be32(ELS_DTAG_CONGESTION); lpfc_printf_vlog(vport, KERN_INFO, LOG_ELS | LOG_CGN_MGMT, - "6444 Xmit RDF to remote NPORT x%x\n", - ndlp->nlp_DID); + "6444 Xmit RDF to remote NPORT x%x Reg: %x %x\n", + ndlp->nlp_DID, phba->cgn_reg_signal, + phba->cgn_reg_fpin); + phba->cgn_fpin_frequency = LPFC_FPIN_INIT_FREQ; elsiocb->iocb_cmpl = lpfc_cmpl_els_disc_cmd; elsiocb->context1 = lpfc_nlp_get(ndlp); if (!elsiocb->context1) { @@ -3778,6 +3782,8 @@ lpfc_least_capable_settings(struct lpfc_hba *phba, { u32 rsp_sig_cap = 0, drv_sig_cap = 0; u32 rsp_sig_freq_cyc = 0, rsp_sig_freq_scale = 0; + struct lpfc_cgn_info *cp; + u16 sig_freq; /* Get rsp signal and frequency capabilities. */ rsp_sig_cap = be32_to_cpu(pcgd->xmt_signal_capability); @@ -3832,6 +3838,24 @@ lpfc_least_capable_settings(struct lpfc_hba *phba, phba->cgn_reg_fpin &= ~LPFC_CGN_FPIN_WARN; } } + + if (!phba->cgn_i) + return; + + /* Update signal frequency in congestion info buffer */ + cp = (struct lpfc_cgn_info *)phba->cgn_i->virt; + + /* Frequency (in ms) Signal Warning/Signal Congestion Notifications + * are received by the HBA + */ + sig_freq = phba->cgn_sig_freq; + + if (phba->cgn_reg_signal == EDC_CG_SIG_WARN_ONLY) + cp->cgn_warn_freq = cpu_to_le16(sig_freq); + if (phba->cgn_reg_signal == EDC_CG_SIG_WARN_ALARM) { + cp->cgn_alarm_freq = cpu_to_le16(sig_freq); + cp->cgn_warn_freq = cpu_to_le16(sig_freq); + } return; out_no_support: @@ -9508,11 +9532,13 @@ lpfc_els_rcv_fpin_peer_cgn(struct lpfc_hba *phba, struct fc_tlv_desc *tlv) static int lpfc_els_rcv_fpin_cgn(struct lpfc_hba *phba, struct fc_tlv_desc *tlv) { + struct lpfc_cgn_info *cp; struct fc_fn_congn_desc *cgn = (struct fc_fn_congn_desc *)tlv; const char *cgn_evt_str; u32 cgn_evt; const char *cgn_sev_str; u32 cgn_sev; + uint16_t value; bool nm_log = false; int rc = 1; @@ -9540,9 +9566,48 @@ lpfc_els_rcv_fpin_cgn(struct lpfc_hba *phba, struct fc_tlv_desc *tlv) switch (cgn_sev) { case FPIN_CONGN_SEVERITY_ERROR: /* Take action here for an Alarm event */ + if (phba->cmf_active_mode != LPFC_CFG_OFF) { + if (phba->cgn_reg_fpin & LPFC_CGN_FPIN_ALARM) { + /* Track of alarm cnt for cgn_info */ + atomic_inc(&phba->cgn_fabric_alarm_cnt); + /* Track of alarm cnt for SYNC_WQE */ + atomic_inc(&phba->cgn_sync_alarm_cnt); + } + goto cleanup; + } break; case FPIN_CONGN_SEVERITY_WARNING: /* Take action here for a Warning event */ + if (phba->cmf_active_mode != LPFC_CFG_OFF) { + if (phba->cgn_reg_fpin & LPFC_CGN_FPIN_WARN) { + /* Track of warning cnt for cgn_info */ + atomic_inc(&phba->cgn_fabric_warn_cnt); + /* Track of warning cnt for SYNC_WQE */ + atomic_inc(&phba->cgn_sync_warn_cnt); + } +cleanup: + /* Save frequency in ms */ + phba->cgn_fpin_frequency = + be32_to_cpu(cgn->event_period); + value = phba->cgn_fpin_frequency; + if (phba->cgn_i) { + cp = (struct lpfc_cgn_info *) + phba->cgn_i->virt; + if (phba->cgn_reg_fpin & + LPFC_CGN_FPIN_ALARM) + cp->cgn_alarm_freq = + cpu_to_le16(value); + if (phba->cgn_reg_fpin & + LPFC_CGN_FPIN_WARN) + cp->cgn_warn_freq = + cpu_to_le16(value); + } + + /* Don't deliver to upper layer since + * driver took action on this tlv. + */ + rc = 0; + } break; } break; diff --git a/drivers/scsi/lpfc/lpfc_hbadisc.c b/drivers/scsi/lpfc/lpfc_hbadisc.c index 95989230b47e..7195ca0275f9 100644 --- a/drivers/scsi/lpfc/lpfc_hbadisc.c +++ b/drivers/scsi/lpfc/lpfc_hbadisc.c @@ -3647,6 +3647,10 @@ lpfc_mbx_cmpl_read_topology(struct lpfc_hba *phba, LPFC_MBOXQ_t *pmb) phba->wait_4_mlo_maint_flg); } lpfc_mbx_process_link_up(phba, la); + + if (phba->cmf_active_mode != LPFC_CFG_OFF) + lpfc_cmf_signal_init(phba); + } else if (attn_type == LPFC_ATT_LINK_DOWN || attn_type == LPFC_ATT_UNEXP_WWPN) { phba->fc_stat.LinkDown++; diff --git a/drivers/scsi/lpfc/lpfc_init.c b/drivers/scsi/lpfc/lpfc_init.c index a34f667e1cd0..65d5ee9b3e30 100644 --- a/drivers/scsi/lpfc/lpfc_init.c +++ b/drivers/scsi/lpfc/lpfc_init.c @@ -93,6 +93,7 @@ static uint32_t lpfc_sli4_enable_intr(struct lpfc_hba *, uint32_t); static void lpfc_sli4_oas_verify(struct lpfc_hba *phba); static uint16_t lpfc_find_cpu_handle(struct lpfc_hba *, uint16_t, int); static void lpfc_setup_bg(struct lpfc_hba *, struct Scsi_Host *); +static int lpfc_sli4_cgn_parm_chg_evt(struct lpfc_hba *); static struct scsi_transport_template *lpfc_transport_template = NULL; static struct scsi_transport_template *lpfc_vport_transport_template = NULL; @@ -3020,6 +3021,123 @@ lpfc_sli4_stop_fcf_redisc_wait_timer(struct lpfc_hba *phba) spin_unlock_irq(&phba->hbalock); } +/** + * lpfc_cmf_stop - Stop CMF processing + * @phba: pointer to lpfc hba data structure. + * + * This is called when the link goes down or if CMF mode is turned OFF. + * It is also called when going offline or unloaded just before the + * congestion info buffer is unregistered. + **/ +void +lpfc_cmf_stop(struct lpfc_hba *phba) +{ + int cpu; + struct lpfc_cgn_stat *cgs; + + /* We only do something if CMF is enabled */ + if (!phba->sli4_hba.pc_sli4_params.cmf) + return; + + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "6221 Stop CMF / Cancel Timer\n"); + + /* Cancel the CMF timer */ + hrtimer_cancel(&phba->cmf_timer); + + /* Zero CMF counters */ + atomic_set(&phba->cmf_busy, 0); + for_each_present_cpu(cpu) { + cgs = per_cpu_ptr(phba->cmf_stat, cpu); + atomic64_set(&cgs->total_bytes, 0); + atomic64_set(&cgs->rcv_bytes, 0); + atomic_set(&cgs->rx_io_cnt, 0); + atomic64_set(&cgs->rx_latency, 0); + } + atomic_set(&phba->cmf_bw_wait, 0); + + /* Resume any blocked IO - Queue unblock on workqueue */ + queue_work(phba->wq, &phba->unblock_request_work); +} + +static inline uint64_t +lpfc_get_max_line_rate(struct lpfc_hba *phba) +{ + uint64_t rate = lpfc_sli_port_speed_get(phba); + + return ((((unsigned long)rate) * 1024 * 1024) / 10); +} + +void +lpfc_cmf_signal_init(struct lpfc_hba *phba) +{ + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "6223 Signal CMF init\n"); + + /* Use the new fc_linkspeed to recalculate */ + phba->cmf_interval_rate = LPFC_CMF_INTERVAL; + phba->cmf_max_line_rate = lpfc_get_max_line_rate(phba); + phba->cmf_link_byte_count = (phba->cmf_max_line_rate * + phba->cmf_interval_rate) / 1000; + phba->cmf_max_bytes_per_interval = phba->cmf_link_byte_count; + + /* This is a signal to firmware to sync up CMF BW with link speed */ + lpfc_issue_cmf_sync_wqe(phba, 0, 0); +} + +/** + * lpfc_cmf_start - Start CMF processing + * @phba: pointer to lpfc hba data structure. + * + * This is called when the link comes up or if CMF mode is turned OFF + * to Monitor or Managed. + **/ +void +lpfc_cmf_start(struct lpfc_hba *phba) +{ + struct lpfc_cgn_stat *cgs; + int cpu; + + /* We only do something if CMF is enabled */ + if (!phba->sli4_hba.pc_sli4_params.cmf || + phba->cmf_active_mode == LPFC_CFG_OFF) + return; + + /* Reinitialize congestion buffer info */ + lpfc_init_congestion_buf(phba); + + atomic_set(&phba->cgn_fabric_warn_cnt, 0); + atomic_set(&phba->cgn_fabric_alarm_cnt, 0); + atomic_set(&phba->cgn_sync_alarm_cnt, 0); + atomic_set(&phba->cgn_sync_warn_cnt, 0); + + atomic_set(&phba->cmf_busy, 0); + for_each_present_cpu(cpu) { + cgs = per_cpu_ptr(phba->cmf_stat, cpu); + atomic64_set(&cgs->total_bytes, 0); + atomic64_set(&cgs->rcv_bytes, 0); + atomic_set(&cgs->rx_io_cnt, 0); + atomic64_set(&cgs->rx_latency, 0); + } + phba->cmf_latency.tv_sec = 0; + phba->cmf_latency.tv_nsec = 0; + + lpfc_cmf_signal_init(phba); + + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "6222 Start CMF / Timer\n"); + + phba->cmf_timer_cnt = 0; + hrtimer_start(&phba->cmf_timer, + ktime_set(0, LPFC_CMF_INTERVAL * 1000000), + HRTIMER_MODE_REL); + /* Setup for latency check in IO cmpl routines */ + ktime_get_real_ts64(&phba->cmf_latency); + + atomic_set(&phba->cmf_bw_wait, 0); + atomic_set(&phba->cmf_stop_io, 0); +} + /** * lpfc_stop_hba_timers - Stop all the timers associated with an HBA * @phba: pointer to lpfc hba data structure. @@ -5286,6 +5404,165 @@ lpfc_async_link_speed_to_read_top(struct lpfc_hba *phba, uint8_t speed_code) return port_speed; } +/** + * lpfc_calc_cmf_latency - latency from start of rxate timer interval + * @phba: The Hba for which this call is being executed. + * + * The routine calculates the latency from the beginning of the CMF timer + * interval to the current point in time. It is called from IO completion + * when we exceed our Bandwidth limitation for the time interval. + */ +uint32_t +lpfc_calc_cmf_latency(struct lpfc_hba *phba) +{ + struct timespec64 cmpl_time; + uint32_t msec = 0; + + ktime_get_real_ts64(&cmpl_time); + + /* This routine works on a ms granularity so sec and usec are + * converted accordingly. + */ + if (cmpl_time.tv_sec == phba->cmf_latency.tv_sec) { + msec = (cmpl_time.tv_nsec - phba->cmf_latency.tv_nsec) / + NSEC_PER_MSEC; + } else { + if (cmpl_time.tv_nsec >= phba->cmf_latency.tv_nsec) { + msec = (cmpl_time.tv_sec - + phba->cmf_latency.tv_sec) * MSEC_PER_SEC; + msec += ((cmpl_time.tv_nsec - + phba->cmf_latency.tv_nsec) / NSEC_PER_MSEC); + } else { + msec = (cmpl_time.tv_sec - phba->cmf_latency.tv_sec - + 1) * MSEC_PER_SEC; + msec += (((NSEC_PER_SEC - phba->cmf_latency.tv_nsec) + + cmpl_time.tv_nsec) / NSEC_PER_MSEC); + } + } + return msec; +} + +/** + * lpfc_cmf_timer - This is the timer function for one congestion + * rate interval. + * @timer: Pointer to the high resolution timer that expired + */ +static enum hrtimer_restart +lpfc_cmf_timer(struct hrtimer *timer) +{ + struct lpfc_hba *phba = container_of(timer, struct lpfc_hba, + cmf_timer); + uint32_t io_cnt; + uint64_t total, rcv, lat, mbpi; + int timer_interval = LPFC_CMF_INTERVAL; + struct lpfc_cgn_stat *cgs; + int cpu; + + /* Only restart the timer if congestion mgmt is on */ + if (phba->cmf_active_mode == LPFC_CFG_OFF || + !phba->cmf_latency.tv_sec) { + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "6224 CMF timer exit: %d %lld\n", + phba->cmf_active_mode, + (uint64_t)phba->cmf_latency.tv_sec); + return HRTIMER_NORESTART; + } + + /* If pport is not ready yet, just exit and wait for + * the next timer cycle to hit. + */ + if (!phba->pport) + goto skip; + + /* Do not block SCSI IO while in the timer routine since + * total_bytes will be cleared + */ + atomic_set(&phba->cmf_stop_io, 1); + + /* Immediately after we calculate the time since the last + * timer interrupt, set the start time for the next + * interrupt + */ + ktime_get_real_ts64(&phba->cmf_latency); + + phba->cmf_link_byte_count = + (phba->cmf_max_line_rate * LPFC_CMF_INTERVAL) / 1000; + + /* Collect all the stats from the prior timer interval */ + total = 0; + io_cnt = 0; + lat = 0; + rcv = 0; + for_each_present_cpu(cpu) { + cgs = per_cpu_ptr(phba->cmf_stat, cpu); + total += atomic64_xchg(&cgs->total_bytes, 0); + io_cnt += atomic_xchg(&cgs->rx_io_cnt, 0); + lat += atomic64_xchg(&cgs->rx_latency, 0); + rcv += atomic64_xchg(&cgs->rcv_bytes, 0); + } + + /* Before we issue another CMF_SYNC_WQE, retrieve the BW + * returned from the last CMF_SYNC_WQE issued, from + * cmf_last_sync_bw. This will be the target BW for + * this next timer interval. + */ + if (phba->cmf_active_mode == LPFC_CFG_MANAGED && + phba->link_state != LPFC_LINK_DOWN && + phba->hba_flag & HBA_SETUP) { + mbpi = phba->cmf_last_sync_bw; + phba->cmf_last_sync_bw = 0; + lpfc_issue_cmf_sync_wqe(phba, LPFC_CMF_INTERVAL, total); + } else { + /* For Monitor mode or link down we want mbpi + * to be the full link speed + */ + mbpi = phba->cmf_link_byte_count; + } + phba->cmf_timer_cnt++; + + if (io_cnt) { + /* Update congestion info buffer latency in us */ + atomic_add(io_cnt, &phba->cgn_latency_evt_cnt); + atomic64_add(lat, &phba->cgn_latency_evt); + } + + /* Calculate MBPI for the next timer interval */ + if (mbpi) { + if (mbpi > phba->cmf_link_byte_count || + phba->cmf_active_mode == LPFC_CFG_MONITOR) + mbpi = phba->cmf_link_byte_count; + + /* Change max_bytes_per_interval to what the prior + * CMF_SYNC_WQE cmpl indicated. + */ + if (mbpi != phba->cmf_max_bytes_per_interval) + phba->cmf_max_bytes_per_interval = mbpi; + } + + if (phba->cmf_active_mode == LPFC_CFG_MONITOR) { + /* If Monitor mode, check if we are oversubscribed + * against the full line rate. + */ + if (mbpi && total > mbpi) + atomic_inc(&phba->cgn_driver_evt_cnt); + } + phba->rx_block_cnt += (rcv / 512); /* save 512 byte block cnt */ + + /* Since total_bytes has already been zero'ed, its okay to unblock + * after max_bytes_per_interval is setup. + */ + if (atomic_xchg(&phba->cmf_bw_wait, 0)) + queue_work(phba->wq, &phba->unblock_request_work); + + /* SCSI IO is now unblocked */ + atomic_set(&phba->cmf_stop_io, 0); + +skip: + hrtimer_forward_now(timer, + ktime_set(0, timer_interval * NSEC_PER_MSEC)); + return HRTIMER_RESTART; +} + #define trunk_link_status(__idx)\ bf_get(lpfc_acqe_fc_la_trunk_config_port##__idx, acqe_fc) ?\ ((phba->trunk_link.link##__idx.state == LPFC_LINK_UP) ?\ @@ -5348,6 +5625,9 @@ lpfc_update_trunk_link_status(struct lpfc_hba *phba, trunk_link_status(0), trunk_link_status(1), trunk_link_status(2), trunk_link_status(3)); + if (phba->cmf_active_mode != LPFC_CFG_OFF) + lpfc_cmf_signal_init(phba); + if (port_fault) lpfc_printf_log(phba, KERN_ERR, LOG_TRACE_EVENT, "3202 trunk error:0x%x (%s) seen on port0:%s " @@ -5688,6 +5968,10 @@ lpfc_sli4_async_sli_evt(struct lpfc_hba *phba, struct lpfc_acqe_sli *acqe_sli) "Event Data1:x%08x Event Data2: x%08x\n", acqe_sli->event_data1, acqe_sli->event_data2); break; + case LPFC_SLI_EVENT_TYPE_PORT_PARAMS_CHG: + /* Call FW to obtain active parms */ + lpfc_sli4_cgn_parm_chg_evt(phba); + break; case LPFC_SLI_EVENT_TYPE_MISCONF_FAWWN: /* Misconfigured WWN. Reports that the SLI Port is configured * to use FA-WWN, but the attached device doesn’t support it. @@ -6113,6 +6397,21 @@ lpfc_sli4_async_grp5_evt(struct lpfc_hba *phba, phba->sli4_hba.link_state.logical_speed); } +/** + * lpfc_sli4_async_cmstat_evt - Process the asynchronous cmstat event + * @phba: pointer to lpfc hba data structure. + * + * This routine is to handle the SLI4 asynchronous cmstat event. A cmstat event + * is an asynchronous notification of a request to reset CM stats. + **/ +static void +lpfc_sli4_async_cmstat_evt(struct lpfc_hba *phba) +{ + if (!phba->cgn_i) + return; + lpfc_init_congestion_stat(phba); +} + /** * lpfc_cgn_params_val - Validate FW congestion parameters. * @phba: pointer to lpfc hba data structure. @@ -6200,6 +6499,7 @@ lpfc_cgn_params_parse(struct lpfc_hba *phba, case LPFC_CFG_OFF: if (phba->cgn_p.cgn_param_mode != LPFC_CFG_OFF) { /* Turning CMF on */ + lpfc_cmf_start(phba); if (phba->link_state >= LPFC_LINK_UP) { phba->cgn_reg_fpin = @@ -6214,6 +6514,7 @@ lpfc_cgn_params_parse(struct lpfc_hba *phba, switch (phba->cgn_p.cgn_param_mode) { case LPFC_CFG_OFF: /* Turning CMF off */ + lpfc_cmf_stop(phba); if (phba->link_state >= LPFC_LINK_UP) lpfc_issue_els_edc(phba->pport, 0); break; @@ -6221,6 +6522,12 @@ lpfc_cgn_params_parse(struct lpfc_hba *phba, lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, "4661 Switch from MANAGED to " "`MONITOR mode\n"); + phba->cmf_max_bytes_per_interval = + phba->cmf_link_byte_count; + + /* Resume blocked IO - unblock on workqueue */ + queue_work(phba->wq, + &phba->unblock_request_work); break; } break; @@ -6228,6 +6535,7 @@ lpfc_cgn_params_parse(struct lpfc_hba *phba, switch (phba->cgn_p.cgn_param_mode) { case LPFC_CFG_OFF: /* Turning CMF off */ + lpfc_cmf_stop(phba); if (phba->link_state >= LPFC_LINK_UP) lpfc_issue_els_edc(phba->pport, 0); break; @@ -6235,6 +6543,7 @@ lpfc_cgn_params_parse(struct lpfc_hba *phba, lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, "4662 Switch from MONITOR to " "MANAGED mode\n"); + lpfc_cmf_signal_init(phba); break; } break; @@ -6301,6 +6610,51 @@ lpfc_sli4_cgn_params_read(struct lpfc_hba *phba) return ret; } +/** + * lpfc_sli4_cgn_parm_chg_evt - Process a FW congestion param change event + * @phba: pointer to lpfc hba data structure. + * + * The FW generated Async ACQE SLI event calls this routine when + * the event type is an SLI Internal Port Event and the Event Code + * indicates a change to the FW maintained congestion parameters. + * + * This routine executes a Read_Object mailbox call to obtain the + * current congestion parameters maintained in FW and corrects + * the driver's active congestion parameters. + * + * The acqe event is not passed because there is no further data + * required. + * + * Returns nonzero error if event processing encountered an error. + * Zero otherwise for success. + **/ +static int +lpfc_sli4_cgn_parm_chg_evt(struct lpfc_hba *phba) +{ + int ret = 0; + + if (!phba->sli4_hba.pc_sli4_params.cmf) { + lpfc_printf_log(phba, KERN_ERR, LOG_CGN_MGMT | LOG_INIT, + "4664 Cgn Evt when E2E off. Drop event\n"); + return -EACCES; + } + + /* If the event is claiming an empty object, it's ok. A write + * could have cleared it. Only error is a negative return + * status. + */ + ret = lpfc_sli4_cgn_params_read(phba); + if (ret < 0) { + lpfc_printf_log(phba, KERN_ERR, LOG_CGN_MGMT | LOG_INIT, + "4667 Error reading Cgn Params (%d)\n", + ret); + } else if (!ret) { + lpfc_printf_log(phba, KERN_ERR, LOG_CGN_MGMT | LOG_INIT, + "4673 CGN Event empty object.\n"); + } + return ret; +} + /** * lpfc_sli4_async_event_proc - Process all the pending asynchronous event * @phba: pointer to lpfc hba data structure. @@ -6349,6 +6703,9 @@ void lpfc_sli4_async_event_proc(struct lpfc_hba *phba) case LPFC_TRAILER_CODE_SLI: lpfc_sli4_async_sli_evt(phba, &cq_event->cqe.acqe_sli); break; + case LPFC_TRAILER_CODE_CMSTAT: + lpfc_sli4_async_cmstat_evt(phba); + break; default: lpfc_printf_log(phba, KERN_ERR, LOG_TRACE_EVENT, @@ -6633,6 +6990,15 @@ lpfc_sli_probe_sriov_nr_virtfn(struct lpfc_hba *phba, int nr_vfn) return rc; } +static void +lpfc_unblock_requests_work(struct work_struct *work) +{ + struct lpfc_hba *phba = container_of(work, struct lpfc_hba, + unblock_request_work); + + lpfc_unblock_requests(phba); +} + /** * lpfc_setup_driver_resource_phase1 - Phase1 etup driver internal resources. * @phba: pointer to lpfc hba data structure. @@ -6708,7 +7074,7 @@ lpfc_setup_driver_resource_phase1(struct lpfc_hba *phba) INIT_DELAYED_WORK(&phba->idle_stat_delay_work, lpfc_idle_stat_delay_work); - + INIT_WORK(&phba->unblock_request_work, lpfc_unblock_requests_work); return 0; } @@ -6939,6 +7305,10 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba) /* FCF rediscover timer */ timer_setup(&phba->fcf.redisc_wait, lpfc_sli4_fcf_redisc_wait_tmo, 0); + /* CMF congestion timer */ + hrtimer_init(&phba->cmf_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL); + phba->cmf_timer.function = lpfc_cmf_timer; + /* * Control structure for handling external multi-buffer mailbox * command pass-through. @@ -7387,6 +7757,14 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba) } #endif + phba->cmf_stat = alloc_percpu(struct lpfc_cgn_stat); + if (!phba->cmf_stat) { + lpfc_printf_log(phba, KERN_ERR, LOG_TRACE_EVENT, + "3331 Failed allocating per cpu cgn stats\n"); + rc = -ENOMEM; + goto out_free_hba_hdwq_info; + } + /* * Enable sr-iov virtual functions if supported and configured * through the module parameter. @@ -7406,6 +7784,8 @@ lpfc_sli4_driver_resource_setup(struct lpfc_hba *phba) return 0; +out_free_hba_hdwq_info: + free_percpu(phba->sli4_hba.c_stat); #ifdef CONFIG_SCSI_LPFC_DEBUG_FS out_free_hba_idle_stat: kfree(phba->sli4_hba.idle_stat); @@ -7453,6 +7833,7 @@ lpfc_sli4_driver_resource_unset(struct lpfc_hba *phba) #ifdef CONFIG_SCSI_LPFC_DEBUG_FS free_percpu(phba->sli4_hba.c_stat); #endif + free_percpu(phba->cmf_stat); kfree(phba->sli4_hba.idle_stat); /* Free memory allocated for msi-x interrupt vector to CPU mapping */ @@ -12352,6 +12733,8 @@ lpfc_sli4_hba_unset(struct lpfc_hba *phba) struct pci_dev *pdev = phba->pcidev; lpfc_stop_hba_timers(phba); + hrtimer_cancel(&phba->cmf_timer); + if (phba->pport) phba->sli4_hba.intr_enable = 0; @@ -12422,7 +12805,6 @@ lpfc_sli4_hba_unset(struct lpfc_hba *phba) phba->pport->work_port_events = 0; } - void lpfc_init_congestion_buf(struct lpfc_hba *phba) { @@ -12519,9 +12901,10 @@ __lpfc_reg_congestion_buf(struct lpfc_hba *phba, int reg) return 0; } -static int +int lpfc_unreg_congestion_buf(struct lpfc_hba *phba) { + lpfc_cmf_stop(phba); return __lpfc_reg_congestion_buf(phba, 0); } @@ -13763,6 +14146,8 @@ lpfc_pci_remove_one_s4(struct pci_dev *pdev) spin_lock_irq(&phba->hbalock); vport->load_flag |= FC_UNLOADING; spin_unlock_irq(&phba->hbalock); + if (phba->cgn_i) + lpfc_unreg_congestion_buf(phba); lpfc_free_sysfs_attr(vport); diff --git a/drivers/scsi/lpfc/lpfc_mem.c b/drivers/scsi/lpfc/lpfc_mem.c index be54fbf5146f..bbb181ab334b 100644 --- a/drivers/scsi/lpfc/lpfc_mem.c +++ b/drivers/scsi/lpfc/lpfc_mem.c @@ -335,6 +335,15 @@ lpfc_mem_free_all(struct lpfc_hba *phba) dma_pool_destroy(phba->lpfc_cmd_rsp_buf_pool); phba->lpfc_cmd_rsp_buf_pool = NULL; + /* Free Congestion Data buffer */ + if (phba->cgn_i) { + dma_free_coherent(&phba->pcidev->dev, + sizeof(struct lpfc_cgn_info), + phba->cgn_i->virt, phba->cgn_i->phys); + kfree(phba->cgn_i); + phba->cgn_i = NULL; + } + /* Free the iocb lookup array */ kfree(psli->iocbq_lookup); psli->iocbq_lookup = NULL; diff --git a/drivers/scsi/lpfc/lpfc_nvme.c b/drivers/scsi/lpfc/lpfc_nvme.c index f36294e9b5dd..73a3568ff17e 100644 --- a/drivers/scsi/lpfc/lpfc_nvme.c +++ b/drivers/scsi/lpfc/lpfc_nvme.c @@ -931,6 +931,8 @@ lpfc_nvme_io_cmd_wqe_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pwqeIn, uint32_t code, status, idx; uint16_t cid, sqhd, data; uint32_t *ptr; + uint32_t lat; + bool call_done = false; #ifdef CONFIG_SCSI_LPFC_DEBUG_FS int cpu; #endif @@ -1135,10 +1137,21 @@ lpfc_nvme_io_cmd_wqe_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pwqeIn, freqpriv = nCmd->private; freqpriv->nvme_buf = NULL; lpfc_ncmd->nvmeCmd = NULL; - spin_unlock(&lpfc_ncmd->buf_lock); + call_done = true; + } + spin_unlock(&lpfc_ncmd->buf_lock); + + /* Check if IO qualified for CMF */ + if (phba->cmf_active_mode != LPFC_CFG_OFF && + nCmd->io_dir == NVMEFC_FCP_READ && + nCmd->payload_length) { + /* Used when calculating average latency */ + lat = ktime_get_ns() - lpfc_ncmd->rx_cmd_start; + lpfc_update_cmf_cmpl(phba, lat, nCmd->payload_length, NULL); + } + + if (call_done) nCmd->done(nCmd); - } else - spin_unlock(&lpfc_ncmd->buf_lock); /* Call release with XB=1 to queue the IO into the abort list. */ lpfc_release_nvme_buf(phba, lpfc_ncmd); @@ -1212,6 +1225,10 @@ lpfc_nvme_prep_io_cmd(struct lpfc_vport *vport, /* Word 5 */ wqe->fcp_iread.rsrvd5 = 0; + /* For a CMF Managed port, iod must be zero'ed */ + if (phba->cmf_active_mode == LPFC_CFG_MANAGED) + bf_set(wqe_iod, &wqe->fcp_iread.wqe_com, + LPFC_WQE_IOD_NONE); cstat->input_requests++; } } else { @@ -1562,6 +1579,19 @@ lpfc_nvme_fcp_io_submit(struct nvme_fc_local_port *pnvme_lport, expedite = 1; } + /* Check if IO qualifies for CMF */ + if (phba->cmf_active_mode != LPFC_CFG_OFF && + pnvme_fcreq->io_dir == NVMEFC_FCP_READ && + pnvme_fcreq->payload_length) { + ret = lpfc_update_cmf_cmd(phba, pnvme_fcreq->payload_length); + if (ret) { + ret = -EBUSY; + goto out_fail; + } + /* Get start time for IO latency */ + start = ktime_get_ns(); + } + /* The node is shared with FCP IO, make sure the IO pending count does * not exceed the programmed depth. */ @@ -1576,7 +1606,7 @@ lpfc_nvme_fcp_io_submit(struct nvme_fc_local_port *pnvme_lport, ndlp->cmd_qdepth); atomic_inc(&lport->xmt_fcp_qdepth); ret = -EBUSY; - goto out_fail; + goto out_fail1; } } @@ -1596,7 +1626,7 @@ lpfc_nvme_fcp_io_submit(struct nvme_fc_local_port *pnvme_lport, "idx %d DID %x\n", lpfc_queue_info->index, ndlp->nlp_DID); ret = -EBUSY; - goto out_fail; + goto out_fail1; } #ifdef CONFIG_SCSI_LPFC_DEBUG_FS if (start) { @@ -1606,6 +1636,7 @@ lpfc_nvme_fcp_io_submit(struct nvme_fc_local_port *pnvme_lport, lpfc_ncmd->ts_cmd_start = 0; } #endif + lpfc_ncmd->rx_cmd_start = start; /* * Store the data needed by the driver to issue, abort, and complete @@ -1687,6 +1718,9 @@ lpfc_nvme_fcp_io_submit(struct nvme_fc_local_port *pnvme_lport, } else cstat->control_requests--; lpfc_release_nvme_buf(phba, lpfc_ncmd); + out_fail1: + lpfc_update_cmf_cmpl(phba, LPFC_CGN_NOT_SENT, + pnvme_fcreq->payload_length, NULL); out_fail: return ret; } diff --git a/drivers/scsi/lpfc/lpfc_scsi.c b/drivers/scsi/lpfc/lpfc_scsi.c index ee4ff4855866..11f26582e68c 100644 --- a/drivers/scsi/lpfc/lpfc_scsi.c +++ b/drivers/scsi/lpfc/lpfc_scsi.c @@ -3853,6 +3853,141 @@ lpfc_scsi_unprep_dma_buf(struct lpfc_hba *phba, struct lpfc_io_buf *psb) psb->pCmd->sc_data_direction); } +/** + * lpfc_unblock_requests - allow further commands to be queued. + * @phba: pointer to phba object + * + * For single vport, just call scsi_unblock_requests on physical port. + * For multiple vports, send scsi_unblock_requests for all the vports. + */ +void +lpfc_unblock_requests(struct lpfc_hba *phba) +{ + struct lpfc_vport **vports; + struct Scsi_Host *shost; + int i; + + if (phba->sli_rev == LPFC_SLI_REV4 && + !phba->sli4_hba.max_cfg_param.vpi_used) { + shost = lpfc_shost_from_vport(phba->pport); + scsi_unblock_requests(shost); + return; + } + + vports = lpfc_create_vport_work_array(phba); + if (vports != NULL) + for (i = 0; i <= phba->max_vports && vports[i] != NULL; i++) { + shost = lpfc_shost_from_vport(vports[i]); + scsi_unblock_requests(shost); + } + lpfc_destroy_vport_work_array(phba, vports); +} + +/** + * lpfc_block_requests - prevent further commands from being queued. + * @phba: pointer to phba object + * + * For single vport, just call scsi_block_requests on physical port. + * For multiple vports, send scsi_block_requests for all the vports. + */ +void +lpfc_block_requests(struct lpfc_hba *phba) +{ + struct lpfc_vport **vports; + struct Scsi_Host *shost; + int i; + + if (atomic_read(&phba->cmf_stop_io)) + return; + + if (phba->sli_rev == LPFC_SLI_REV4 && + !phba->sli4_hba.max_cfg_param.vpi_used) { + shost = lpfc_shost_from_vport(phba->pport); + scsi_block_requests(shost); + return; + } + + vports = lpfc_create_vport_work_array(phba); + if (vports != NULL) + for (i = 0; i <= phba->max_vports && vports[i] != NULL; i++) { + shost = lpfc_shost_from_vport(vports[i]); + scsi_block_requests(shost); + } + lpfc_destroy_vport_work_array(phba, vports); +} + +/** + * lpfc_update_cmf_cmpl - Adjust CMF counters for IO completion + * @phba: The HBA for which this call is being executed. + * @time: The latency of the IO that completed (in ns) + * @size: The size of the IO that completed + * @shost: SCSI host the IO completed on (NULL for a NVME IO) + * + * The routine adjusts the various Burst and Bandwidth counters used in + * Congestion management and E2E. If time is set to LPFC_CGN_NOT_SENT, + * that means the IO was never issued to the HBA, so this routine is + * just being called to cleanup the counter from a previous + * lpfc_update_cmf_cmd call. + */ +int +lpfc_update_cmf_cmpl(struct lpfc_hba *phba, + uint64_t time, uint32_t size, struct Scsi_Host *shost) +{ + struct lpfc_cgn_stat *cgs; + + if (time != LPFC_CGN_NOT_SENT) { + /* lat is ns coming in, save latency in us */ + if (time < 1000) + time = 1; + else + time = (time + 500) / 1000; /* round it */ + + cgs = this_cpu_ptr(phba->cmf_stat); + atomic64_add(size, &cgs->rcv_bytes); + atomic64_add(time, &cgs->rx_latency); + atomic_inc(&cgs->rx_io_cnt); + } + return 0; +} + +/** + * lpfc_update_cmf_cmd - Adjust CMF counters for IO submission + * @phba: The HBA for which this call is being executed. + * @size: The size of the IO that will be issued + * + * The routine adjusts the various Burst and Bandwidth counters used in + * Congestion management and E2E. + */ +int +lpfc_update_cmf_cmd(struct lpfc_hba *phba, uint32_t size) +{ + uint64_t total; + struct lpfc_cgn_stat *cgs; + int cpu; + + /* At this point we are either LPFC_CFG_MANAGED or LPFC_CFG_MONITOR */ + if (phba->cmf_active_mode == LPFC_CFG_MANAGED) { + total = 0; + for_each_present_cpu(cpu) { + cgs = per_cpu_ptr(phba->cmf_stat, cpu); + total += atomic64_read(&cgs->total_bytes); + } + if (total >= phba->cmf_max_bytes_per_interval) { + if (!atomic_xchg(&phba->cmf_bw_wait, 1)) { + lpfc_block_requests(phba); + phba->cmf_last_ts = + lpfc_calc_cmf_latency(phba); + } + atomic_inc(&phba->cmf_busy); + return -EBUSY; + } + } + + cgs = this_cpu_ptr(phba->cmf_stat); + atomic64_add(size, &cgs->total_bytes); + return 0; +} + /** * lpfc_handle_fcp_err - FCP response handler * @vport: The virtual port for which this call is being executed. @@ -4063,6 +4198,7 @@ lpfc_fcp_io_cmd_wqe_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pwqeIn, u32 logit = LOG_FCP; u32 status, idx; unsigned long iflags = 0; + u32 lat; u8 wait_xb_clr = 0; /* Sanity check on return of outstanding command */ @@ -4351,10 +4487,21 @@ lpfc_fcp_io_cmd_wqe_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pwqeIn, lpfc_io_ktime(phba, lpfc_cmd); } #endif + if (likely(!wait_xb_clr)) + lpfc_cmd->pCmd = NULL; + spin_unlock(&lpfc_cmd->buf_lock); + + /* Check if IO qualified for CMF */ + if (phba->cmf_active_mode != LPFC_CFG_OFF && + cmd->sc_data_direction == DMA_FROM_DEVICE && + (scsi_sg_count(cmd))) { + /* Used when calculating average latency */ + lat = ktime_get_ns() - lpfc_cmd->rx_cmd_start; + lpfc_update_cmf_cmpl(phba, lat, scsi_bufflen(cmd), shost); + } + if (wait_xb_clr) goto out; - lpfc_cmd->pCmd = NULL; - spin_unlock(&lpfc_cmd->buf_lock); /* The sdev is not guaranteed to be valid post scsi_done upcall. */ cmd->scsi_done(cmd); @@ -4367,8 +4514,8 @@ lpfc_fcp_io_cmd_wqe_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *pwqeIn, lpfc_cmd->cur_iocbq.iocb_flag &= ~LPFC_DRIVER_ABORTED; if (lpfc_cmd->waitq) wake_up(lpfc_cmd->waitq); -out: spin_unlock(&lpfc_cmd->buf_lock); +out: lpfc_release_scsi_buf(phba, lpfc_cmd); } @@ -4775,6 +4922,11 @@ static int lpfc_scsi_prep_cmnd_buf_s4(struct lpfc_vport *vport, fcp_cmnd->fcpCntl3 = READ_DATA; if (hdwq) hdwq->scsi_cstat.input_requests++; + + /* For a CMF Managed port, iod must be zero'ed */ + if (phba->cmf_active_mode == LPFC_CFG_MANAGED) + bf_set(wqe_iod, &wqe->fcp_iread.wqe_com, + LPFC_WQE_IOD_NONE); } } else { /* From the icmnd template, initialize words 4 - 11 */ @@ -5458,7 +5610,7 @@ lpfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) if (phba->ktime_on) start = ktime_get_ns(); #endif - + start = ktime_get_ns(); rdata = lpfc_rport_data_from_scsi_device(cmnd->device); /* sanity check on references */ @@ -5489,7 +5641,18 @@ lpfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) * transport is still transitioning. */ if (!ndlp) - goto out_tgt_busy; + goto out_tgt_busy1; + + /* Check if IO qualifies for CMF */ + if (phba->cmf_active_mode != LPFC_CFG_OFF && + cmnd->sc_data_direction == DMA_FROM_DEVICE && + (scsi_sg_count(cmnd))) { + /* Latency start time saved in rx_cmd_start later in routine */ + err = lpfc_update_cmf_cmd(phba, scsi_bufflen(cmnd)); + if (err) + goto out_tgt_busy1; + } + if (lpfc_ndlp_check_qdepth(phba, ndlp)) { if (atomic_read(&ndlp->cmd_pending) >= ndlp->cmd_qdepth) { lpfc_printf_vlog(vport, KERN_INFO, LOG_FCP_ERROR, @@ -5517,7 +5680,7 @@ lpfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) ndlp->nlp_portname.u.wwn[5], ndlp->nlp_portname.u.wwn[6], ndlp->nlp_portname.u.wwn[7]); - goto out_tgt_busy; + goto out_tgt_busy2; } } @@ -5530,6 +5693,7 @@ lpfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) "IO busied\n"); goto out_host_busy; } + lpfc_cmd->rx_cmd_start = start; /* * Store the midlayer's command structure for the completion phase @@ -5674,13 +5838,20 @@ lpfc_queuecommand(struct Scsi_Host *shost, struct scsi_cmnd *cmnd) out_host_busy_release_buf: lpfc_release_scsi_buf(phba, lpfc_cmd); out_host_busy: + lpfc_update_cmf_cmpl(phba, LPFC_CGN_NOT_SENT, scsi_bufflen(cmnd), + shost); return SCSI_MLQUEUE_HOST_BUSY; - out_tgt_busy: + out_tgt_busy2: + lpfc_update_cmf_cmpl(phba, LPFC_CGN_NOT_SENT, scsi_bufflen(cmnd), + shost); + out_tgt_busy1: return SCSI_MLQUEUE_TARGET_BUSY; out_fail_command_release_buf: lpfc_release_scsi_buf(phba, lpfc_cmd); + lpfc_update_cmf_cmpl(phba, LPFC_CGN_NOT_SENT, scsi_bufflen(cmnd), + shost); out_fail_command: cmnd->scsi_done(cmnd); diff --git a/drivers/scsi/lpfc/lpfc_sli.c b/drivers/scsi/lpfc/lpfc_sli.c index 4d1c190823d1..6d256a509ea3 100644 --- a/drivers/scsi/lpfc/lpfc_sli.c +++ b/drivers/scsi/lpfc/lpfc_sli.c @@ -1439,7 +1439,7 @@ __lpfc_sli_release_iocbq_s4(struct lpfc_hba *phba, struct lpfc_iocbq *iocbq) memset((char *)iocbq + start_clean, 0, sizeof(*iocbq) - start_clean); iocbq->sli4_lxritag = NO_XRI; iocbq->sli4_xritag = NO_XRI; - iocbq->iocb_flag &= ~(LPFC_IO_NVME | LPFC_IO_NVMET | + iocbq->iocb_flag &= ~(LPFC_IO_NVME | LPFC_IO_NVMET | LPFC_IO_CMF | LPFC_IO_NVME_LS); list_add_tail(&iocbq->list, &phba->lpfc_iocb_list); } @@ -1785,9 +1785,11 @@ lpfc_cmf_sync_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, { union lpfc_wqe128 *wqe; uint32_t status, info; - uint64_t bw; + uint64_t bw, bwdif, slop; + uint64_t pcent, bwpcent; int asig, afpin, sigcnt, fpincnt; - int cg, tdp; + int wsigmax, wfpinmax, cg, tdp; + char *s; /* First check for error */ status = bf_get(lpfc_wcqe_c_status, cmf_cmpl); @@ -1806,6 +1808,14 @@ lpfc_cmf_sync_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, /* Gather congestion information on a successful cmpl */ info = cmf_cmpl->parameter; + phba->cmf_active_info = info; + + /* See if firmware info count is valid or has changed */ + if (info > LPFC_MAX_CMF_INFO || phba->cmf_info_per_interval == info) + info = 0; + else + phba->cmf_info_per_interval = info; + tdp = bf_get(lpfc_wcqe_c_cmf_bw, cmf_cmpl); cg = bf_get(lpfc_wcqe_c_cmf_cg, cmf_cmpl); @@ -1824,7 +1834,65 @@ lpfc_cmf_sync_cmpl(struct lpfc_hba *phba, struct lpfc_iocbq *cmdiocb, afpin = bf_get(cmf_sync_afpin, &wqe->cmf_sync); fpincnt = bf_get(cmf_sync_wfpincnt, &wqe->cmf_sync); sigcnt = bf_get(cmf_sync_wsigcnt, &wqe->cmf_sync); + if (phba->cmf_max_bytes_per_interval != bw || + (asig || afpin || sigcnt || fpincnt)) { + /* Are we increasing or decreasing BW */ + if (phba->cmf_max_bytes_per_interval < bw) { + bwdif = bw - phba->cmf_max_bytes_per_interval; + s = "Increase"; + } else { + bwdif = phba->cmf_max_bytes_per_interval - bw; + s = "Decrease"; + } + + /* What is the change percentage */ + slop = (phba->cmf_link_byte_count / 200); /* For rounding */ + pcent = ((bwdif * 100) + slop) / phba->cmf_link_byte_count; + bwpcent = ((bw * 100) + slop) / phba->cmf_link_byte_count; + if (asig) { + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "6237 BW Threshold %lld%% (%lld): " + "%lld%% %s: Signal Alarm: cg:%d " + "Info:%u\n", + bwpcent, bw, pcent, s, cg, + phba->cmf_active_info); + } else if (afpin) { + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "6238 BW Threshold %lld%% (%lld): " + "%lld%% %s: FPIN Alarm: cg:%d " + "Info:%u\n", + bwpcent, bw, pcent, s, cg, + phba->cmf_active_info); + } else if (sigcnt) { + wsigmax = bf_get(cmf_sync_wsigmax, &wqe->cmf_sync); + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "6239 BW Threshold %lld%% (%lld): " + "%lld%% %s: Signal Warning: " + "Cnt %d Max %d: cg:%d Info:%u\n", + bwpcent, bw, pcent, s, sigcnt, + wsigmax, cg, phba->cmf_active_info); + } else if (fpincnt) { + wfpinmax = bf_get(cmf_sync_wfpinmax, &wqe->cmf_sync); + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "6240 BW Threshold %lld%% (%lld): " + "%lld%% %s: FPIN Warning: " + "Cnt %d Max %d: cg:%d Info:%u\n", + bwpcent, bw, pcent, s, fpincnt, + wfpinmax, cg, phba->cmf_active_info); + } else { + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "6241 BW Threshold %lld%% (%lld): " + "CMF %lld%% %s: cg:%d Info:%u\n", + bwpcent, bw, pcent, s, cg, + phba->cmf_active_info); + } + } else if (info) { + lpfc_printf_log(phba, KERN_INFO, LOG_CGN_MGMT, + "6246 Info Threshold %u\n", info); + } + /* Save BW change to be picked up during next timer interrupt */ + phba->cmf_last_sync_bw = bw; out: lpfc_sli_release_iocbq(phba, cmdiocb); } @@ -4645,6 +4713,7 @@ lpfc_sli_brdready_s4(struct lpfc_hba *phba, uint32_t mask) } else phba->sli4_hba.intr_enable = 0; + phba->hba_flag &= ~HBA_SETUP; return retval; } @@ -4965,6 +5034,7 @@ lpfc_sli4_brdreset(struct lpfc_hba *phba) phba->link_events = 0; phba->pport->fc_myDID = 0; phba->pport->fc_prevDID = 0; + phba->hba_flag &= ~HBA_SETUP; spin_lock_irq(&phba->hbalock); psli->sli_flag &= ~(LPFC_PROCESS_LA); @@ -6663,8 +6733,14 @@ lpfc_set_features(struct lpfc_hba *phba, LPFC_MBOXQ_t *mbox, bf_set(lpfc_mbx_set_feature_mi, &mbox->u.mqe.un.set_feature, phba->sli4_hba.pc_sli4_params.mi_ver); break; + case LPFC_SET_ENABLE_CMF: + bf_set(lpfc_mbx_set_feature_dd, &mbox->u.mqe.un.set_feature, 1); + mbox->u.mqe.un.set_feature.feature = LPFC_SET_ENABLE_CMF; + mbox->u.mqe.un.set_feature.param_len = 4; + bf_set(lpfc_mbx_set_feature_cmf, + &mbox->u.mqe.un.set_feature, 1); + break; } - return; } @@ -7813,10 +7889,11 @@ lpfc_cmf_setup(struct lpfc_hba *phba) { LPFC_MBOXQ_t *mboxq; struct lpfc_mqe *mqe; + struct lpfc_dmabuf *mp; struct lpfc_pc_sli4_params *sli4_params; struct lpfc_sli4_parameters *mbx_sli4_parameters; int length; - int rc, mi_ver; + int rc, cmf, mi_ver; mboxq = (LPFC_MBOXQ_t *)mempool_alloc(phba->mbox_mem_pool, GFP_KERNEL); if (!mboxq) @@ -7836,10 +7913,11 @@ lpfc_cmf_setup(struct lpfc_hba *phba) return rc; } - /* Gather info on MI support */ + /* Gather info on CMF and MI support */ sli4_params = &phba->sli4_hba.pc_sli4_params; mbx_sli4_parameters = &mqe->un.get_sli4_parameters.sli4_parameters; sli4_params->mi_ver = bf_get(cfg_mi_ver, mbx_sli4_parameters); + sli4_params->cmf = bf_get(cfg_cmf, mbx_sli4_parameters); /* Are we forcing MI off via module parameter? */ if (!phba->cfg_enable_mi) @@ -7886,9 +7964,95 @@ lpfc_cmf_setup(struct lpfc_hba *phba) if (sli4_params->mi_ver) phba->cfg_fdmi_on = LPFC_FDMI_SUPPORT; + /* Always try to enable CMF feature if we can */ + if (sli4_params->cmf) { + lpfc_set_features(phba, mboxq, LPFC_SET_ENABLE_CMF); + rc = lpfc_sli_issue_mbox(phba, mboxq, MBX_POLL); + cmf = bf_get(lpfc_mbx_set_feature_cmf, + &mboxq->u.mqe.un.set_feature); + if (rc == MBX_SUCCESS && cmf) { + lpfc_printf_log(phba, KERN_WARNING, LOG_CGN_MGMT, + "6218 CMF is enabled: mode %d\n", + phba->cmf_active_mode); + } else { + lpfc_printf_log(phba, KERN_WARNING, + LOG_CGN_MGMT | LOG_INIT, + "6219 Enable CMF Mailbox x%x (x%x/x%x) " + "failed, rc:x%x dd:x%x\n", + bf_get(lpfc_mqe_command, &mboxq->u.mqe), + lpfc_sli_config_mbox_subsys_get + (phba, mboxq), + lpfc_sli_config_mbox_opcode_get + (phba, mboxq), + rc, cmf); + sli4_params->cmf = 0; + phba->cmf_active_mode = LPFC_CFG_OFF; + goto no_cmf; + } + + /* Allocate Congestion Information Buffer */ + if (!phba->cgn_i) { + mp = kmalloc(sizeof(*mp), GFP_KERNEL); + if (mp) + mp->virt = dma_alloc_coherent + (&phba->pcidev->dev, + sizeof(struct lpfc_cgn_info), + &mp->phys, GFP_KERNEL); + if (!mp || !mp->virt) { + lpfc_printf_log(phba, KERN_ERR, LOG_INIT, + "2640 Failed to alloc memory " + "for Congestion Info\n"); + kfree(mp); + sli4_params->cmf = 0; + phba->cmf_active_mode = LPFC_CFG_OFF; + goto no_cmf; + } + phba->cgn_i = mp; + + /* initialize congestion buffer info */ + lpfc_init_congestion_buf(phba); + lpfc_init_congestion_stat(phba); + } + + rc = lpfc_sli4_cgn_params_read(phba); + if (rc < 0) { + lpfc_printf_log(phba, KERN_ERR, LOG_CGN_MGMT | LOG_INIT, + "6242 Error reading Cgn Params (%d)\n", + rc); + /* Ensure CGN Mode is off */ + sli4_params->cmf = 0; + } else if (!rc) { + lpfc_printf_log(phba, KERN_ERR, LOG_CGN_MGMT | LOG_INIT, + "6243 CGN Event empty object.\n"); + /* Ensure CGN Mode is off */ + sli4_params->cmf = 0; + } + } else { +no_cmf: + lpfc_printf_log(phba, KERN_WARNING, LOG_CGN_MGMT, + "6220 CMF is disabled\n"); + } + + /* Only register congestion buffer with firmware if BOTH + * CMF and E2E are enabled. + */ + if (sli4_params->cmf && sli4_params->mi_ver) { + rc = lpfc_reg_congestion_buf(phba); + if (rc) { + dma_free_coherent(&phba->pcidev->dev, + sizeof(struct lpfc_cgn_info), + phba->cgn_i->virt, phba->cgn_i->phys); + kfree(phba->cgn_i); + phba->cgn_i = NULL; + /* Ensure CGN Mode is off */ + phba->cmf_active_mode = LPFC_CFG_OFF; + return 0; + } + } lpfc_printf_log(phba, KERN_INFO, LOG_INIT, - "6470 Setup MI version %d\n", - sli4_params->mi_ver); + "6470 Setup MI version %d CMF %d mode %d\n", + sli4_params->mi_ver, sli4_params->cmf, + phba->cmf_active_mode); mempool_free(mboxq, phba->mbox_mem_pool); @@ -7901,6 +8065,7 @@ lpfc_cmf_setup(struct lpfc_hba *phba) atomic_set(&phba->cgn_latency_evt_cnt, 0); atomic64_set(&phba->cgn_latency_evt, 0); + phba->cmf_interval_rate = LPFC_CMF_INTERVAL; return 0; } @@ -8042,8 +8207,6 @@ lpfc_sli4_hba_setup(struct lpfc_hba *phba) lpfc_printf_log(phba, KERN_ERR, LOG_MBOX | LOG_INIT, "6468 Set host date / time: Status x%x:\n", rc); - lpfc_cmf_setup(phba); - /* * Continue initialization with default values even if driver failed * to read FCoE param config regions, only read parameters if the @@ -8571,6 +8734,9 @@ lpfc_sli4_hba_setup(struct lpfc_hba *phba) /* Indicate device interrupt mode */ phba->sli4_hba.intr_enable = 1; + /* Setup CMF after HBA is initialized */ + lpfc_cmf_setup(phba); + if (!(phba->hba_flag & HBA_FCOE_MODE) && (phba->hba_flag & LINK_DISABLED)) { lpfc_printf_log(phba, KERN_ERR, LOG_TRACE_EVENT, @@ -8592,7 +8758,10 @@ lpfc_sli4_hba_setup(struct lpfc_hba *phba) } } mempool_free(mboxq, phba->mbox_mem_pool); + + phba->hba_flag |= HBA_SETUP; return rc; + out_io_buff_free: /* Free allocated IO Buffers */ lpfc_io_free(phba); @@ -21104,8 +21273,7 @@ lpfc_sli4_issue_wqe(struct lpfc_hba *phba, struct lpfc_sli4_hdw_queue *qp, } /* NVME_FCREQ and NVME_ABTS requests */ - if (pwqe->iocb_flag & LPFC_IO_NVME || - pwqe->iocb_flag & LPFC_IO_FCP) { + if (pwqe->iocb_flag & (LPFC_IO_NVME | LPFC_IO_FCP | LPFC_IO_CMF)) { /* Get the IO distribution (hba_wqidx) for WQ assignment. */ wq = qp->io_wq; pring = wq->pring; diff --git a/drivers/scsi/lpfc/lpfc_sli.h b/drivers/scsi/lpfc/lpfc_sli.h index dc7cc2f37089..5161ccacea3e 100644 --- a/drivers/scsi/lpfc/lpfc_sli.h +++ b/drivers/scsi/lpfc/lpfc_sli.h @@ -463,4 +463,5 @@ struct lpfc_io_buf { uint64_t ts_isr_cmpl; uint64_t ts_data_io; #endif + uint64_t rx_cmd_start; }; diff --git a/drivers/scsi/lpfc/lpfc_sli4.h b/drivers/scsi/lpfc/lpfc_sli4.h index f250b666ac57..99c5d1e4da5e 100644 --- a/drivers/scsi/lpfc/lpfc_sli4.h +++ b/drivers/scsi/lpfc/lpfc_sli4.h @@ -557,6 +557,7 @@ struct lpfc_pc_sli4_params { uint16_t mi_value; #define LPFC_DFLT_MIB_VAL 2 uint8_t mib_bde_cnt; + uint8_t cmf; uint8_t cqv; uint8_t mqv; uint8_t wqv; From patchwork Fri Aug 13 23:00:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 496856 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3BDB8C4338F for ; Fri, 13 Aug 2021 23:01:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 241F16109E for ; Fri, 13 Aug 2021 23:01:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235428AbhHMXBw (ORCPT ); Fri, 13 Aug 2021 19:01:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41620 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235425AbhHMXBq (ORCPT ); Fri, 13 Aug 2021 19:01:46 -0400 Received: from mail-pl1-x632.google.com (mail-pl1-x632.google.com [IPv6:2607:f8b0:4864:20::632]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3DE56C061756 for ; Fri, 13 Aug 2021 16:01:19 -0700 (PDT) Received: by mail-pl1-x632.google.com with SMTP id d17so13887594plr.12 for ; Fri, 13 Aug 2021 16:01:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=W3K6/7PGor0E176ZzywrNUq7xPALmPPlfrZgqNRtEDY=; b=MlczA4/Y2Bxg2m++BYx8YuIbru6xfJEakxxd4AcnhgiLVkWiY/SMg0KNGa3N7ezHuZ btDr1t/fQ6hwhJcmlJzClPtr+WFxJqUhDJ32HG1dA2L06WyZh4PRNWoS8ifha1YtKv3y k/62jUDsEDJNz20ZKA+tmO01RyuzofOajodeFOj7ekcF8Jl5OAJ7mvWufJ/SZGAUUNYZ xW48PlU5sZCQJy9FDmh1n2my8S0HcUJgYs+jQonKm5kidRSxVASlNdJaIR1KmmosVqIy DE5Kcuj+jgzftf2mOy1O/seQL2XCQPIyYNeqLH2LK9T1Vho4d2wBEWF7HgDb6kodUdd+ 2fTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=W3K6/7PGor0E176ZzywrNUq7xPALmPPlfrZgqNRtEDY=; b=mGBosx1zAUHrNoT/5rSi7T502KHiI0bSOqGaX7lUGuvy02kaHlHArjcPI47m4PU4Mk c7IJzxOsI96lsOFBiSEckzpjnDG5PtyH9j0JowlBMVLTsckdmVoP4X5TWvPsXZjr4N9N 9bLGFNYJOSQ+aKGCwBQ/HoW7G57K1kKPArpxkcF46Qen5hXSG7fjs6a1SRm+AQbBu6W6 BWutE6PaJcCc3HTGq1IYvi37stjXcOQKezmzyy3AHx5vjUHoVQMG+CmWSP8UZKYV7s1r pFRJZ1UoGB8I8mgD+uMqy4TLf8K24igI+fVycYaFR1L64wilJTfXEJGy1rDuk/iRftRF OJnA== X-Gm-Message-State: AOAM532F6LZUgIoaLqs1XPxnEIn3Y+poqyxCbyqFep7zKz0A1QkkFkUj 5TLwS2K154rWAf6JZ1bDCPS7IFXmCT0= X-Google-Smtp-Source: ABdhPJw71bc5LCsfRLzOweb/oTxAEd1GrOJVZI/ZpbwQDE4C3+docy9ftT3sSkWMWYFQxOTRiXjfDA== X-Received: by 2002:a17:902:8f90:b029:12d:1b48:efd8 with SMTP id z16-20020a1709028f90b029012d1b48efd8mr3773762plo.23.1628895678694; Fri, 13 Aug 2021 16:01:18 -0700 (PDT) Received: from localhost.localdomain.localdomain ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id e8sm4001997pgg.31.2021.08.13.16.01.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 16:01:18 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH v2 12/16] lpfc: Add debugfs support for cm framework buffers Date: Fri, 13 Aug 2021 16:00:35 -0700 Message-Id: <20210813230039.110546-13-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210813230039.110546-1-jsmart2021@gmail.com> References: <20210813230039.110546-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org This patch adds support via debugfs to report the cm statistics, cm enablement, and rx monitor information. Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc.h | 2 + drivers/scsi/lpfc/lpfc_debugfs.c | 223 +++++++++++++++++++++++++++++++ drivers/scsi/lpfc/lpfc_debugfs.h | 9 ++ 3 files changed, 234 insertions(+) diff --git a/drivers/scsi/lpfc/lpfc.h b/drivers/scsi/lpfc/lpfc.h index 640075885540..dd8cb111b199 100644 --- a/drivers/scsi/lpfc/lpfc.h +++ b/drivers/scsi/lpfc/lpfc.h @@ -1357,6 +1357,8 @@ struct lpfc_hba { #ifdef LPFC_HDWQ_LOCK_STAT struct dentry *debug_lockstat; #endif + struct dentry *debug_cgn_buffer; + struct dentry *debug_rx_monitor; struct dentry *debug_ras_log; atomic_t nvmeio_trc_cnt; uint32_t nvmeio_trc_size; diff --git a/drivers/scsi/lpfc/lpfc_debugfs.c b/drivers/scsi/lpfc/lpfc_debugfs.c index 6ff85ae57e79..bd6d459afce5 100644 --- a/drivers/scsi/lpfc/lpfc_debugfs.c +++ b/drivers/scsi/lpfc/lpfc_debugfs.c @@ -5429,6 +5429,180 @@ lpfc_idiag_extacc_read(struct file *file, char __user *buf, size_t nbytes, return simple_read_from_buffer(buf, nbytes, ppos, pbuffer, len); } +static int +lpfc_cgn_buffer_open(struct inode *inode, struct file *file) +{ + struct lpfc_debug *debug; + int rc = -ENOMEM; + + debug = kmalloc(sizeof(*debug), GFP_KERNEL); + if (!debug) + goto out; + + debug->buffer = vmalloc(LPFC_CGN_BUF_SIZE); + if (!debug->buffer) { + kfree(debug); + goto out; + } + + debug->i_private = inode->i_private; + file->private_data = debug; + + rc = 0; +out: + return rc; +} + +static ssize_t +lpfc_cgn_buffer_read(struct file *file, char __user *buf, size_t nbytes, + loff_t *ppos) +{ + struct lpfc_debug *debug = file->private_data; + struct lpfc_hba *phba = (struct lpfc_hba *)debug->i_private; + char *buffer = debug->buffer; + uint32_t *ptr; + int cnt, len = 0; + + if (!phba->sli4_hba.pc_sli4_params.mi_ver || !phba->cgn_i) { + len += scnprintf(buffer + len, LPFC_CGN_BUF_SIZE - len, + "Congestion Mgmt is not supported\n"); + goto out; + } + ptr = (uint32_t *)phba->cgn_i->virt; + len += scnprintf(buffer + len, LPFC_CGN_BUF_SIZE - len, + "Congestion Buffer Header\n"); + /* Dump the first 32 bytes */ + cnt = 32; + len += scnprintf(buffer + len, LPFC_CGN_BUF_SIZE - len, + "000: %08x %08x %08x %08x %08x %08x %08x %08x\n", + *ptr, *(ptr + 1), *(ptr + 2), *(ptr + 3), + *(ptr + 4), *(ptr + 5), *(ptr + 6), *(ptr + 7)); + ptr += 8; + len += scnprintf(buffer + len, LPFC_CGN_BUF_SIZE - len, + "Congestion Buffer Data\n"); + while (cnt < sizeof(struct lpfc_cgn_info)) { + if (len > (LPFC_CGN_BUF_SIZE - LPFC_DEBUG_OUT_LINE_SZ)) { + len += scnprintf(buffer + len, LPFC_CGN_BUF_SIZE - len, + "Truncated . . .\n"); + break; + } + len += scnprintf(buffer + len, LPFC_CGN_BUF_SIZE - len, + "%03x: %08x %08x %08x %08x " + "%08x %08x %08x %08x\n", + cnt, *ptr, *(ptr + 1), *(ptr + 2), + *(ptr + 3), *(ptr + 4), *(ptr + 5), + *(ptr + 6), *(ptr + 7)); + cnt += 32; + ptr += 8; + } +out: + return simple_read_from_buffer(buf, nbytes, ppos, buffer, len); +} + +static int +lpfc_cgn_buffer_release(struct inode *inode, struct file *file) +{ + struct lpfc_debug *debug = file->private_data; + + vfree(debug->buffer); + kfree(debug); + + return 0; +} + +static int +lpfc_rx_monitor_open(struct inode *inode, struct file *file) +{ + struct lpfc_rx_monitor_debug *debug; + int rc = -ENOMEM; + + debug = kmalloc(sizeof(*debug), GFP_KERNEL); + if (!debug) + goto out; + + debug->buffer = vmalloc(MAX_DEBUGFS_RX_TABLE_SIZE); + if (!debug->buffer) { + kfree(debug); + goto out; + } + + debug->i_private = inode->i_private; + file->private_data = debug; + + rc = 0; +out: + return rc; +} + +static ssize_t +lpfc_rx_monitor_read(struct file *file, char __user *buf, size_t nbytes, + loff_t *ppos) +{ + struct lpfc_rx_monitor_debug *debug = file->private_data; + struct lpfc_hba *phba = (struct lpfc_hba *)debug->i_private; + char *buffer = debug->buffer; + struct rxtable_entry *entry; + int i, len = 0, head, tail, last, start; + + head = atomic_read(&phba->rxtable_idx_head); + while (head == LPFC_RXMONITOR_TABLE_IN_USE) { + /* Table is getting updated */ + msleep(20); + head = atomic_read(&phba->rxtable_idx_head); + } + + tail = atomic_xchg(&phba->rxtable_idx_tail, head); + if (!phba->rxtable || head == tail) { + len += scnprintf(buffer + len, MAX_DEBUGFS_RX_TABLE_SIZE - len, + "Rxtable is empty\n"); + goto out; + } + last = (head > tail) ? head : LPFC_MAX_RXMONITOR_ENTRY; + start = tail; + + len += scnprintf(buffer + len, MAX_DEBUGFS_RX_TABLE_SIZE - len, + " MaxBPI\t Total Data Cmd Total Data Cmpl " + " Latency(us) Avg IO Size\tMax IO Size IO cnt " + "Info BWutil(ms)\n"); +get_table: + for (i = start; i < last; i++) { + entry = &phba->rxtable[i]; + len += scnprintf(buffer + len, MAX_DEBUGFS_RX_TABLE_SIZE - len, + "%3d:%12lld %12lld\t%12lld\t" + "%8lldus\t%8lld\t%10lld " + "%8d %2d %2d(%2d)\n", + i, entry->max_bytes_per_interval, + entry->total_bytes, + entry->rcv_bytes, + entry->avg_io_latency, + entry->avg_io_size, + entry->max_read_cnt, + entry->io_cnt, + entry->cmf_info, + entry->timer_utilization, + entry->timer_interval); + } + + if (head != last) { + start = 0; + last = head; + goto get_table; + } +out: + return simple_read_from_buffer(buf, nbytes, ppos, buffer, len); +} + +static int +lpfc_rx_monitor_release(struct inode *inode, struct file *file) +{ + struct lpfc_rx_monitor_debug *debug = file->private_data; + + vfree(debug->buffer); + kfree(debug); + + return 0; +} + #undef lpfc_debugfs_op_disc_trc static const struct file_operations lpfc_debugfs_op_disc_trc = { .owner = THIS_MODULE, @@ -5657,6 +5831,23 @@ static const struct file_operations lpfc_idiag_op_extAcc = { .write = lpfc_idiag_extacc_write, .release = lpfc_idiag_cmd_release, }; +#undef lpfc_cgn_buffer_op +static const struct file_operations lpfc_cgn_buffer_op = { + .owner = THIS_MODULE, + .open = lpfc_cgn_buffer_open, + .llseek = lpfc_debugfs_lseek, + .read = lpfc_cgn_buffer_read, + .release = lpfc_cgn_buffer_release, +}; + +#undef lpfc_rx_monitor_op +static const struct file_operations lpfc_rx_monitor_op = { + .owner = THIS_MODULE, + .open = lpfc_rx_monitor_open, + .llseek = lpfc_debugfs_lseek, + .read = lpfc_rx_monitor_read, + .release = lpfc_rx_monitor_release, +}; #endif /* lpfc_idiag_mbxacc_dump_bsg_mbox - idiag debugfs dump bsg mailbox command @@ -5907,6 +6098,32 @@ lpfc_debugfs_initialize(struct lpfc_vport *vport) goto debug_failed; } + /* Congestion Info Buffer */ + scnprintf(name, sizeof(name), "cgn_buffer"); + phba->debug_cgn_buffer = + debugfs_create_file(name, S_IFREG | 0644, + phba->hba_debugfs_root, + phba, &lpfc_cgn_buffer_op); + if (!phba->debug_cgn_buffer) { + lpfc_printf_vlog(vport, KERN_ERR, LOG_INIT, + "6527 Cannot create debugfs " + "cgn_buffer\n"); + goto debug_failed; + } + + /* RX Monitor */ + scnprintf(name, sizeof(name), "rx_monitor"); + phba->debug_rx_monitor = + debugfs_create_file(name, S_IFREG | 0644, + phba->hba_debugfs_root, + phba, &lpfc_rx_monitor_op); + if (!phba->debug_rx_monitor) { + lpfc_printf_vlog(vport, KERN_ERR, LOG_INIT, + "6528 Cannot create debugfs " + "rx_monitor\n"); + goto debug_failed; + } + /* RAS log */ snprintf(name, sizeof(name), "ras_log"); phba->debug_ras_log = @@ -6335,6 +6552,12 @@ lpfc_debugfs_terminate(struct lpfc_vport *vport) debugfs_remove(phba->debug_hbqinfo); /* hbqinfo */ phba->debug_hbqinfo = NULL; + debugfs_remove(phba->debug_cgn_buffer); + phba->debug_cgn_buffer = NULL; + + debugfs_remove(phba->debug_rx_monitor); + phba->debug_rx_monitor = NULL; + debugfs_remove(phba->debug_ras_log); phba->debug_ras_log = NULL; diff --git a/drivers/scsi/lpfc/lpfc_debugfs.h b/drivers/scsi/lpfc/lpfc_debugfs.h index 7ab6d3b08698..dd4cdd8563eb 100644 --- a/drivers/scsi/lpfc/lpfc_debugfs.h +++ b/drivers/scsi/lpfc/lpfc_debugfs.h @@ -52,6 +52,9 @@ /* scsistat output buffer size */ #define LPFC_SCSISTAT_SIZE 8192 +/* Congestion Info Buffer size */ +#define LPFC_CGN_BUF_SIZE 8192 + #define LPFC_DEBUG_OUT_LINE_SZ 80 /* @@ -279,6 +282,12 @@ struct lpfc_idiag { void *ptr_private; }; +#define MAX_DEBUGFS_RX_TABLE_SIZE (100 * LPFC_MAX_RXMONITOR_ENTRY) +struct lpfc_rx_monitor_debug { + char *i_private; + char *buffer; +}; + #else #define lpfc_nvmeio_data(phba, fmt, arg...) \ From patchwork Fri Aug 13 23:00:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 496855 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7CD5AC432BE for ; Fri, 13 Aug 2021 23:01:32 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 65728610CF for ; Fri, 13 Aug 2021 23:01:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235440AbhHMXB7 (ORCPT ); Fri, 13 Aug 2021 19:01:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41654 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235362AbhHMXBy (ORCPT ); Fri, 13 Aug 2021 19:01:54 -0400 Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DE0E6C061756 for ; Fri, 13 Aug 2021 16:01:26 -0700 (PDT) Received: by mail-pj1-x102a.google.com with SMTP id fa24-20020a17090af0d8b0290178bfa69d97so18199202pjb.0 for ; Fri, 13 Aug 2021 16:01:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Rll7Qj4dW4CpMS9diLJDYM5fmrrZYHUz76jCCwqBRPw=; b=DMFolXv35ipRKcH2nnaANgDZ3WnpSCpe2CDijr5mDh7IHU8qFj9Mz92TbrUB/b+OAh 291pzuNANL+YsNrCVEKTEM2cGEKtQnmwecjjsbc/3IaZ8vIywtqi6BdquyJ8WtxFewir KiW+wpY9DKUStlqTr3je6CB6iMNEMwp4tY3mPf3PEwzzj4aRD6DbaomRTIOjZlVmCRbJ 7G+v7ZXmb8cUq1vKXJaWlk1jE75AwcUSC8O9E2jaqMEJlkKtxM+om3tkWBuVDmnUmMwJ lQ7ByprJd3cbNvrcKy80X8wV9/K/R0Ju8VoBtM9KZr1ZKGlDstIYa8cvVUWyZkBTkbVt fX5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Rll7Qj4dW4CpMS9diLJDYM5fmrrZYHUz76jCCwqBRPw=; b=VEZfIsfj5QAb6qPAUz0CUr+4Yl2UmW45UUKXo4xo7Mtq9Tx8zKCw2KfQyxU9nly0rR kMijRM3FeQW1SxNYigYwTHnVhftAvmnp1Qk9F+pAs3CCHeLp0L1Vfr0yNeVt/8XoyTG1 gyKtesWFwTJ2MbU6tiiPyuC5nn1ftzj/Gz7Vvx4TfE8HhGNLeohrzq2s8/Z8XpknTOc7 kUj8aiC1n+nfilIDzuYZr+P0yACDygJF3RLofK9a/C1egSdlgPxuCSkiBRKmigJ0H2eM L/yCIdc8ZbDvTlsrHTBijhcZg3sa9QCdTXTRZsDl3n5NIIqESPw2UAVoT2apdksTuYO9 KXGw== X-Gm-Message-State: AOAM531iivyqP3kqqTV7jyv1MSbetd3/gXVgeZlBfIDFeSK4OD7s4NDy x+kcgZw8mHIcBiUARvC0xKgWIziL7GU= X-Google-Smtp-Source: ABdhPJxhZ1+l9Hqk6r6POZMQNt1N7Zkk4QjM+keSp3Q8SlX6Qfc/ltSf6A6edlNQ4H+Ysxc5vIIRCQ== X-Received: by 2002:a17:90a:458c:: with SMTP id v12mr4889716pjg.50.1628895686456; Fri, 13 Aug 2021 16:01:26 -0700 (PDT) Received: from localhost.localdomain.localdomain ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id e8sm4001997pgg.31.2021.08.13.16.01.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 16:01:26 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH v2 15/16] lpfc: Update lpfc version to 14.0.0.1 Date: Fri, 13 Aug 2021 16:00:38 -0700 Message-Id: <20210813230039.110546-16-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210813230039.110546-1-jsmart2021@gmail.com> References: <20210813230039.110546-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Update lpfc version to 14.0.0.1 Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_version.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/scsi/lpfc/lpfc_version.h b/drivers/scsi/lpfc/lpfc_version.h index 73a5b3bbdacd..a7aba7833425 100644 --- a/drivers/scsi/lpfc/lpfc_version.h +++ b/drivers/scsi/lpfc/lpfc_version.h @@ -20,7 +20,7 @@ * included with this package. * *******************************************************************/ -#define LPFC_DRIVER_VERSION "14.0.0.0" +#define LPFC_DRIVER_VERSION "14.0.0.1" #define LPFC_DRIVER_NAME "lpfc" /* Used for SLI 2/3 */ From patchwork Fri Aug 13 23:00:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: James Smart X-Patchwork-Id: 496854 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DAA0C432BE for ; Fri, 13 Aug 2021 23:01:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6FD586109E for ; Fri, 13 Aug 2021 23:01:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235504AbhHMXCB (ORCPT ); Fri, 13 Aug 2021 19:02:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41664 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235498AbhHMXB4 (ORCPT ); Fri, 13 Aug 2021 19:01:56 -0400 Received: from mail-pl1-x634.google.com (mail-pl1-x634.google.com [IPv6:2607:f8b0:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 75718C0617AF for ; Fri, 13 Aug 2021 16:01:29 -0700 (PDT) Received: by mail-pl1-x634.google.com with SMTP id a20so13994881plm.0 for ; Fri, 13 Aug 2021 16:01:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AsxydOY+lNRb2e3ADR2di+F/3cNRDqMO4+kiJOm870I=; b=PW9kHWxVTVKr+4czkc2exMMHdtXShVcgRiXuj8Eq1qjoQffy9HN/b3AHUVnoaqmSMy 6DG/qs3r/gvPYiUg4eMxc5vbmoq486TKyrTvL85LijcF8rCsiQXENQ44pXMvz7sqrjCZ qZulP9FI7L9Y6ZnHLjBZRfSmSJCxHfNjSdabvTEhtt8YoK2VpM6UeW5kn2++RMGty6eH 5GdB0J2ERUiH5elY4ygx71e3KjlBm/UlM7sbVf8epzHSfZBfw1TVR1YIrPXMe4iZnbuJ sWungw6Vu//BAtcRc+CSoIAwx8BeeJBKW51v0K/Y6KGeCr64PVV0rBTgvC/Fh+SJqJiF IMyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AsxydOY+lNRb2e3ADR2di+F/3cNRDqMO4+kiJOm870I=; b=UkHOBc0DyclIbK5Y+fNlK2weu3w8RojWDF9BgYxDhtXW8uxdLPzsm9neHa78RT0S0m xfSMnQOQ4xwJSAv94R6xij/71ARX0JhNSe1mcMHPkEhPLb9tLG9adtMLMQDlOFTBmNDg /taAg66HTFkwhtiCZbCy8XGzdyCXJ5KZSj0FHmRSnmOYLH23kjGP+3XoquBL9n0PA3Lx v0sRbyAyz8ZTM9B+X3AJXfZAMeGirGQBOrI4VZk9PovO2Th+LuZblaslomMgUEI7SPdE 9mXuaBj20grSRAkK4LBBGKBJbc7fIrQ3mcWL0cIb46GhE0gWWeSy166n8t6WmItaMMYh RX0g== X-Gm-Message-State: AOAM531b/H2M9f6vzVvcg1deRAR0zzBgFNGQcmLPKJ5x/KnFsBf4MS9/ PEMx2Jrftf7Z06UZj/kGIDBhLfAJXW8= X-Google-Smtp-Source: ABdhPJyU/IrI8R1EIvgVG5nHRO+baAN6iJ4ew0lECzYdoMG9mT/h/VL7bQbl5f7N7NNK/3+cRK4aug== X-Received: by 2002:a17:902:e20a:b029:12d:76cd:6721 with SMTP id u10-20020a170902e20ab029012d76cd6721mr3747221plb.43.1628895688961; Fri, 13 Aug 2021 16:01:28 -0700 (PDT) Received: from localhost.localdomain.localdomain ([192.19.223.252]) by smtp.gmail.com with ESMTPSA id e8sm4001997pgg.31.2021.08.13.16.01.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 13 Aug 2021 16:01:28 -0700 (PDT) From: James Smart To: linux-scsi@vger.kernel.org Cc: James Smart , Justin Tee Subject: [PATCH v2 16/16] lpfc: Copyright updates for 14.0.0.1 patches Date: Fri, 13 Aug 2021 16:00:39 -0700 Message-Id: <20210813230039.110546-17-jsmart2021@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210813230039.110546-1-jsmart2021@gmail.com> References: <20210813230039.110546-1-jsmart2021@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Update copyrights to 2021 for files modified in the 14.0.0.1 patch set. Co-developed-by: Justin Tee Signed-off-by: Justin Tee Signed-off-by: James Smart --- drivers/scsi/lpfc/lpfc_bsg.h | 2 +- drivers/scsi/lpfc/lpfc_debugfs.h | 2 +- drivers/scsi/lpfc/lpfc_logmsg.h | 2 +- drivers/scsi/lpfc/lpfc_mem.c | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/scsi/lpfc/lpfc_bsg.h b/drivers/scsi/lpfc/lpfc_bsg.h index 17012bcc7c38..749d6c43cfce 100644 --- a/drivers/scsi/lpfc/lpfc_bsg.h +++ b/drivers/scsi/lpfc/lpfc_bsg.h @@ -1,7 +1,7 @@ /******************************************************************* * This file is part of the Emulex Linux Device Driver for * * Fibre Channel Host Bus Adapters. * - * Copyright (C) 2017-2019 Broadcom. All Rights Reserved. The term * + * Copyright (C) 2017-2021 Broadcom. All Rights Reserved. The term * * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. * * Copyright (C) 2010-2015 Emulex. All rights reserved. * * EMULEX and SLI are trademarks of Emulex. * diff --git a/drivers/scsi/lpfc/lpfc_debugfs.h b/drivers/scsi/lpfc/lpfc_debugfs.h index dd4cdd8563eb..a5bf71b34972 100644 --- a/drivers/scsi/lpfc/lpfc_debugfs.h +++ b/drivers/scsi/lpfc/lpfc_debugfs.h @@ -1,7 +1,7 @@ /******************************************************************* * This file is part of the Emulex Linux Device Driver for * * Fibre Channel Host Bus Adapters. * - * Copyright (C) 2017-2019 Broadcom. All Rights Reserved. The term * + * Copyright (C) 2017-2021 Broadcom. All Rights Reserved. The term * * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. * * Copyright (C) 2007-2011 Emulex. All rights reserved. * * EMULEX and SLI are trademarks of Emulex. * diff --git a/drivers/scsi/lpfc/lpfc_logmsg.h b/drivers/scsi/lpfc/lpfc_logmsg.h index d719a16c0f96..7d480c798794 100644 --- a/drivers/scsi/lpfc/lpfc_logmsg.h +++ b/drivers/scsi/lpfc/lpfc_logmsg.h @@ -1,7 +1,7 @@ /******************************************************************* * This file is part of the Emulex Linux Device Driver for * * Fibre Channel Host Bus Adapters. * - * Copyright (C) 2017-2018 Broadcom. All Rights Reserved. The term * + * Copyright (C) 2017-2021 Broadcom. All Rights Reserved. The term * * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. * * Copyright (C) 2004-2009 Emulex. All rights reserved. * * EMULEX and SLI are trademarks of Emulex. * diff --git a/drivers/scsi/lpfc/lpfc_mem.c b/drivers/scsi/lpfc/lpfc_mem.c index 7cb9f4b52b49..870e53b8f81d 100644 --- a/drivers/scsi/lpfc/lpfc_mem.c +++ b/drivers/scsi/lpfc/lpfc_mem.c @@ -1,7 +1,7 @@ /******************************************************************* * This file is part of the Emulex Linux Device Driver for * * Fibre Channel Host Bus Adapters. * - * Copyright (C) 2017-2018 Broadcom. All Rights Reserved. The term * + * Copyright (C) 2017-2021 Broadcom. All Rights Reserved. The term * * “Broadcom” refers to Broadcom Inc. and/or its subsidiaries. * * Copyright (C) 2004-2014 Emulex. All rights reserved. * * EMULEX and SLI are trademarks of Emulex. *