From patchwork Wed May 17 22:23:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 683848 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DDE15C7EE2A for ; Wed, 17 May 2023 22:24:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229517AbjEQWYK (ORCPT ); Wed, 17 May 2023 18:24:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60280 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229445AbjEQWYI (ORCPT ); Wed, 17 May 2023 18:24:08 -0400 Received: from mail-pj1-f41.google.com (mail-pj1-f41.google.com [209.85.216.41]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 152C31B5 for ; Wed, 17 May 2023 15:24:08 -0700 (PDT) Received: by mail-pj1-f41.google.com with SMTP id 98e67ed59e1d1-24e01ba9e03so1092211a91.1 for ; Wed, 17 May 2023 15:24:08 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684362247; x=1686954247; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dZcwxtfcD8Fe17tSglc1QdlV0KH+SGvkZJrbqOzBVcA=; b=RJah39kltf2uVzh39ujDAhOaFB60nMo0uK2bvIqzwzIBsFEhpphW5AMCfq5PCyI7o/ hocCnghRfLqgbrBJgy6S3O4soY/6ZO6vrW/PO6GwLE/dc19ooSBewACnGSE2fF9PT0sR iwFhW1aMW8Ap0gjy1aBvZYPMmq3BI/TXO3z/y+j4JkWdYk8vt6s6e037rOd+uKe6lMfg QHqgRBfOz9XAplibHLcNvsKyu61PpD54QXYyRjDwipnZoQ51OSbuZbim+NO0zL+xjUtL vbTk0ogZQjfejH5yd7sBT7KVbFsOcVMCVs6zLcq8lxIlGwZUCZ2wZ6jf5EZWkIq3ytt3 HaLw== X-Gm-Message-State: AC+VfDzuVyTmLrxAblsaK0JsPdrbgFGY3KPpZL83ytIf2nCViiaEO6yg 2SgBuKLRvIT6/m8iw+9Ciqs= X-Google-Smtp-Source: ACHHUZ7TLJXlWfVrI/tIqx8zezz17hTm6e2OGlbhngL55fiflq8atuf/Z9qlR4IMdd0oAzjZNnjXAQ== X-Received: by 2002:a17:90a:ee94:b0:247:6022:9595 with SMTP id i20-20020a17090aee9400b0024760229595mr322447pjz.45.1684362247487; Wed, 17 May 2023 15:24:07 -0700 (PDT) Received: from bvanassche-glaptop2.roam.corp.google.com ([98.51.102.78]) by smtp.gmail.com with ESMTPSA id d32-20020a17090a6f2300b0024df6bbf5d8sm66273pjk.30.2023.05.17.15.24.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 15:24:07 -0700 (PDT) From: Bart Van Assche To: "Martin K . Petersen" Cc: linux-scsi@vger.kernel.org, Bart Van Assche , Mike Christie Subject: [PATCH v2 2/4] scsi: core: Support setting BLK_MQ_F_BLOCKING Date: Wed, 17 May 2023 15:23:57 -0700 Message-ID: <20230517222359.1066918-3-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.698.g37aff9b760-goog In-Reply-To: <20230517222359.1066918-1-bvanassche@acm.org> References: <20230517222359.1066918-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Prepare for adding code in ufshcd_queuecommand() that may sleep. This patch is similar to a patch posted last year by Mike Christie. See also https://lore.kernel.org/all/20220308003957.123312-2-michael.christie@oracle.com/ Reviewed-by: Mike Christie Signed-off-by: Bart Van Assche --- drivers/scsi/scsi_lib.c | 10 +++------- include/scsi/scsi_host.h | 3 +++ 2 files changed, 6 insertions(+), 7 deletions(-) diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index 758a57616dd3..894af68babc2 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -1982,6 +1982,8 @@ int scsi_mq_setup_tags(struct Scsi_Host *shost) tag_set->flags = BLK_MQ_F_SHOULD_MERGE; tag_set->flags |= BLK_ALLOC_POLICY_TO_MQ_FLAG(shost->hostt->tag_alloc_policy); + if (shost->hostt->queuecommand_may_block) + tag_set->flags |= BLK_MQ_F_BLOCKING; tag_set->driver_data = shost; if (shost->host_tagset) tag_set->flags |= BLK_MQ_F_TAG_HCTX_SHARED; @@ -2968,13 +2970,7 @@ scsi_host_block(struct Scsi_Host *shost) } } - /* - * SCSI never enables blk-mq's BLK_MQ_F_BLOCKING flag so - * calling synchronize_rcu() once is enough. - */ - WARN_ON_ONCE(shost->tag_set.flags & BLK_MQ_F_BLOCKING); - - synchronize_rcu(); + blk_mq_wait_quiesce_done(&shost->tag_set); return 0; } diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h index 0f29799efa02..37a8a2608dc2 100644 --- a/include/scsi/scsi_host.h +++ b/include/scsi/scsi_host.h @@ -458,6 +458,9 @@ struct scsi_host_template { /* True if the host uses host-wide tagspace */ unsigned host_tagset:1; + /* The queuecommand callback may block. See also BLK_MQ_F_BLOCKING. */ + unsigned queuecommand_may_block:1; + /* * Countdown for host blocking with no commands outstanding. */ From patchwork Wed May 17 22:31:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 683845 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC39AC7EE22 for ; Wed, 17 May 2023 22:32:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229560AbjEQWcO (ORCPT ); Wed, 17 May 2023 18:32:14 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35216 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229475AbjEQWcI (ORCPT ); Wed, 17 May 2023 18:32:08 -0400 Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8427D2112 for ; Wed, 17 May 2023 15:32:06 -0700 (PDT) Received: by mail-pg1-f175.google.com with SMTP id 41be03b00d2f7-5307502146aso904110a12.1 for ; Wed, 17 May 2023 15:32:06 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684362726; x=1686954726; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=nlqmL7TqYg7KAdV8g+R4LitmgpZaCm8VAciGH7EULQU=; b=C1RSCDDQWsjOKHAjIQlI0kPI63QoUqooVNpl3gDC37/QKi8pd/Cb0jsJJptZK5H/VD QVTZjdefyeAgYLK2p2Y0W5KI8VjY9Sy/phe4lNA/6hlr31XU/PRL3VkDn3KWAlHf3DEu DjVt8dbfAqAYdwXu2bIVXJrURzrYEew/aGoq6bmHRTTbE0WqnG+weOnLemNE6dUDGuPh cvn2jIn16YcrN/y8ZMgoaySM4+fs5E2wMw8kt6ERNp06rke6lr+sorK7kvtL1l1txBeB TfNnO9NeG/4ocLAIq3vpatr764iJWYOfJlchAu644IkCmEZyEHiWhXo+wgxvs8qokPER cGew== X-Gm-Message-State: AC+VfDz1lBcufhj8auM5oPZMS2Wfbw6WFnCr7hDQwAEVMO0qmbDLLPS9 nxOAtc3snTh9/UALxJ7ZEQAWHPLZk4U= X-Google-Smtp-Source: ACHHUZ5aOJCnjY8njPA6qYn0qdMecilWxx86uZNqsYWntoq7T50Q3HGF4TZlNeRnyZvsMAyPS44Kqw== X-Received: by 2002:a17:90a:d505:b0:24e:46ee:e30d with SMTP id t5-20020a17090ad50500b0024e46eee30dmr459441pju.15.1684362725854; Wed, 17 May 2023 15:32:05 -0700 (PDT) Received: from bvanassche-glaptop2.roam.corp.google.com ([98.51.102.78]) by smtp.gmail.com with ESMTPSA id n1-20020a17090a9f0100b00250d908a771sm61938pjp.50.2023.05.17.15.32.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 15:32:05 -0700 (PDT) From: Bart Van Assche To: "Martin K . Petersen" Cc: linux-scsi@vger.kernel.org, Bart Van Assche Subject: [PATCH v2 3/4] scsi: ufs: Move ufshcd_wl_shutdown() Date: Wed, 17 May 2023 15:31:56 -0700 Message-ID: <20230517223157.1068210-4-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.698.g37aff9b760-goog In-Reply-To: <20230517223157.1068210-1-bvanassche@acm.org> References: <20230517223157.1068210-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Move the definition of ufshcd_wl_shutdown() to make the next patch in this series easier to review. Signed-off-by: Bart Van Assche Reviewed-by: Adrian Hunter --- drivers/ufs/core/ufshcd.c | 44 +++++++++++++++++++-------------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 68d9e24fac98..0f426d46d91e 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -9746,28 +9746,6 @@ static int ufshcd_wl_resume(struct device *dev) } #endif -static void ufshcd_wl_shutdown(struct device *dev) -{ - struct scsi_device *sdev = to_scsi_device(dev); - struct ufs_hba *hba; - - hba = shost_priv(sdev->host); - - down(&hba->host_sem); - hba->shutting_down = true; - up(&hba->host_sem); - - /* Turn on everything while shutting down */ - ufshcd_rpm_get_sync(hba); - scsi_device_quiesce(sdev); - shost_for_each_device(sdev, hba->host) { - if (sdev == hba->ufs_device_wlun) - continue; - scsi_device_quiesce(sdev); - } - __ufshcd_wl_suspend(hba, UFS_SHUTDOWN_PM); -} - /** * ufshcd_suspend - helper function for suspend operations * @hba: per adapter instance @@ -9952,6 +9930,28 @@ int ufshcd_runtime_resume(struct device *dev) EXPORT_SYMBOL(ufshcd_runtime_resume); #endif /* CONFIG_PM */ +static void ufshcd_wl_shutdown(struct device *dev) +{ + struct scsi_device *sdev = to_scsi_device(dev); + struct ufs_hba *hba; + + hba = shost_priv(sdev->host); + + down(&hba->host_sem); + hba->shutting_down = true; + up(&hba->host_sem); + + /* Turn on everything while shutting down */ + ufshcd_rpm_get_sync(hba); + scsi_device_quiesce(sdev); + shost_for_each_device(sdev, hba->host) { + if (sdev == hba->ufs_device_wlun) + continue; + scsi_device_quiesce(sdev); + } + __ufshcd_wl_suspend(hba, UFS_SHUTDOWN_PM); +} + /** * ufshcd_shutdown - shutdown routine * @hba: per adapter instance From patchwork Wed May 17 22:23:59 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 683847 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E776C7EE22 for ; Wed, 17 May 2023 22:24:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229574AbjEQWYP (ORCPT ); Wed, 17 May 2023 18:24:15 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60328 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229555AbjEQWYM (ORCPT ); Wed, 17 May 2023 18:24:12 -0400 Received: from mail-pj1-f50.google.com (mail-pj1-f50.google.com [209.85.216.50]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C10EB46B7 for ; Wed, 17 May 2023 15:24:09 -0700 (PDT) Received: by mail-pj1-f50.google.com with SMTP id 98e67ed59e1d1-24dfc3c662eso1078808a91.3 for ; Wed, 17 May 2023 15:24:09 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684362249; x=1686954249; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oUwBuRk7pox9AkpSoyVSkR/AKy8700rD/uBt+n9cbCU=; b=OtfcTu8jNXItjC7H9WN1ATGmaoI/j7U009ynfp090h1ultZ6dtAXhPguztjMQtKDx1 UCsn4iVjJhWDBr1/SbySkLsT13BaHfxOdI9vLgZLaJZKI1zyjfwq5Rx7lTGtkAJbqcGT sP0EmzIcWG56Q2v9SMvMVF+7KdEAkiqjMvj33JbWP0n+KHH0mbOcos8q/SwlPCvMpjd7 CUgbA6eQvMGhpF0qwC4/7HOxHry2XnupR5SZIUXk3g4isq3970YDqO6ZtMn6+BXpZS1u 0PoSNJ6uyWm2X7qmdZ38MAo2UgQpZmtPmyPtkF+H25upBYFkQs8LbzNjUGMAtQgM2ajb H0Gw== X-Gm-Message-State: AC+VfDwjyaYN7y/oZIAbmKWg3xAJHTR+W08qRK56PRZckI2yX1Eic+m1 QIssYRGFyVkNVc9wlDpGzH93pKXFAC8= X-Google-Smtp-Source: ACHHUZ4VvxQ0pouMot/Y7nu/709HzIeizPtq5uOm63+CiOI5/hDmNiGEeniVPdwnQsuXsX1ghWEn2Q== X-Received: by 2002:a17:90a:a015:b0:24e:120e:30ff with SMTP id q21-20020a17090aa01500b0024e120e30ffmr386203pjp.14.1684362249144; Wed, 17 May 2023 15:24:09 -0700 (PDT) Received: from bvanassche-glaptop2.roam.corp.google.com ([98.51.102.78]) by smtp.gmail.com with ESMTPSA id d32-20020a17090a6f2300b0024df6bbf5d8sm66273pjk.30.2023.05.17.15.24.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 17 May 2023 15:24:08 -0700 (PDT) From: Bart Van Assche To: "Martin K . Petersen" Cc: linux-scsi@vger.kernel.org, Bart Van Assche Subject: [PATCH v2 4/4] scsi: ufs: Ungate the clock synchronously Date: Wed, 17 May 2023 15:23:59 -0700 Message-ID: <20230517222359.1066918-5-bvanassche@acm.org> X-Mailer: git-send-email 2.40.1.698.g37aff9b760-goog In-Reply-To: <20230517222359.1066918-1-bvanassche@acm.org> References: <20230517222359.1066918-1-bvanassche@acm.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Ungating the clock asynchronously causes ufshcd_queuecommand() to return SCSI_MLQUEUE_HOST_BUSY and hence causes commands to be requeued. This is suboptimal. Allow ufshcd_queuecommand() to sleep such that clock ungating does not trigger command requeuing. Remove the ufshcd_scsi_block_requests() and ufshcd_scsi_unblock_requests() because these are no longer needed. The flush_work(&hba->clk_gating.ungate_work) call is sufficient to make the SCSI core wait for clock ungating to complete. Signed-off-by: Bart Van Assche --- drivers/ufs/core/ufs-sysfs.c | 2 +- drivers/ufs/core/ufshcd-crypto.c | 2 +- drivers/ufs/core/ufshcd-priv.h | 2 +- drivers/ufs/core/ufshcd.c | 84 ++++++++++---------------------- include/ufs/ufshcd.h | 2 +- 5 files changed, 30 insertions(+), 62 deletions(-) diff --git a/drivers/ufs/core/ufs-sysfs.c b/drivers/ufs/core/ufs-sysfs.c index 883f0e44b54e..cdf3d5f2b77b 100644 --- a/drivers/ufs/core/ufs-sysfs.c +++ b/drivers/ufs/core/ufs-sysfs.c @@ -168,7 +168,7 @@ static ssize_t auto_hibern8_show(struct device *dev, } pm_runtime_get_sync(hba->dev); - ufshcd_hold(hba, false); + ufshcd_hold(hba); ahit = ufshcd_readl(hba, REG_AUTO_HIBERNATE_IDLE_TIMER); ufshcd_release(hba); pm_runtime_put_sync(hba->dev); diff --git a/drivers/ufs/core/ufshcd-crypto.c b/drivers/ufs/core/ufshcd-crypto.c index 198360fe5e8e..f2c4422cab86 100644 --- a/drivers/ufs/core/ufshcd-crypto.c +++ b/drivers/ufs/core/ufshcd-crypto.c @@ -24,7 +24,7 @@ static int ufshcd_program_key(struct ufs_hba *hba, u32 slot_offset = hba->crypto_cfg_register + slot * sizeof(*cfg); int err = 0; - ufshcd_hold(hba, false); + ufshcd_hold(hba); if (hba->vops && hba->vops->program_key) { err = hba->vops->program_key(hba, cfg, slot); diff --git a/drivers/ufs/core/ufshcd-priv.h b/drivers/ufs/core/ufshcd-priv.h index d53b93c21a0c..22cac71090ae 100644 --- a/drivers/ufs/core/ufshcd-priv.h +++ b/drivers/ufs/core/ufshcd-priv.h @@ -84,7 +84,7 @@ unsigned long ufshcd_mcq_poll_cqe_lock(struct ufs_hba *hba, int ufshcd_read_string_desc(struct ufs_hba *hba, u8 desc_index, u8 **buf, bool ascii); -int ufshcd_hold(struct ufs_hba *hba, bool async); +void ufshcd_hold(struct ufs_hba *hba); void ufshcd_release(struct ufs_hba *hba); int ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd); diff --git a/drivers/ufs/core/ufshcd.c b/drivers/ufs/core/ufshcd.c index 993034ac1696..9736b2b4120e 100644 --- a/drivers/ufs/core/ufshcd.c +++ b/drivers/ufs/core/ufshcd.c @@ -1189,7 +1189,7 @@ static int ufshcd_wait_for_doorbell_clr(struct ufs_hba *hba, bool timeout = false, do_last_check = false; ktime_t start; - ufshcd_hold(hba, false); + ufshcd_hold(hba); spin_lock_irqsave(hba->host->host_lock, flags); /* * Wait for all the outstanding tasks/transfer requests. @@ -1310,7 +1310,7 @@ static int ufshcd_clock_scaling_prepare(struct ufs_hba *hba, u64 timeout_us) } /* let's not get into low power until clock scaling is completed */ - ufshcd_hold(hba, false); + ufshcd_hold(hba); out: return ret; @@ -1640,7 +1640,7 @@ static ssize_t ufshcd_clkscale_enable_store(struct device *dev, goto out; ufshcd_rpm_get_sync(hba); - ufshcd_hold(hba, false); + ufshcd_hold(hba); hba->clk_scaling.is_enabled = value; @@ -1723,7 +1723,7 @@ static void ufshcd_ungate_work(struct work_struct *work) spin_lock_irqsave(hba->host->host_lock, flags); if (hba->clk_gating.state == CLKS_ON) { spin_unlock_irqrestore(hba->host->host_lock, flags); - goto unblock_reqs; + return; } spin_unlock_irqrestore(hba->host->host_lock, flags); @@ -1746,25 +1746,21 @@ static void ufshcd_ungate_work(struct work_struct *work) } hba->clk_gating.is_suspended = false; } -unblock_reqs: - ufshcd_scsi_unblock_requests(hba); } /** * ufshcd_hold - Enable clocks that were gated earlier due to ufshcd_release. * Also, exit from hibern8 mode and set the link as active. * @hba: per adapter instance - * @async: This indicates whether caller should ungate clocks asynchronously. */ -int ufshcd_hold(struct ufs_hba *hba, bool async) +void ufshcd_hold(struct ufs_hba *hba) { - int rc = 0; bool flush_result; unsigned long flags; if (!ufshcd_is_clkgating_allowed(hba) || !hba->clk_gating.is_initialized) - goto out; + return; spin_lock_irqsave(hba->host->host_lock, flags); hba->clk_gating.active_reqs++; @@ -1781,15 +1777,10 @@ int ufshcd_hold(struct ufs_hba *hba, bool async) */ if (ufshcd_can_hibern8_during_gating(hba) && ufshcd_is_link_hibern8(hba)) { - if (async) { - rc = -EAGAIN; - hba->clk_gating.active_reqs--; - break; - } spin_unlock_irqrestore(hba->host->host_lock, flags); flush_result = flush_work(&hba->clk_gating.ungate_work); if (hba->clk_gating.is_suspended && !flush_result) - goto out; + return; spin_lock_irqsave(hba->host->host_lock, flags); goto start; } @@ -1811,21 +1802,14 @@ int ufshcd_hold(struct ufs_hba *hba, bool async) hba->clk_gating.state = REQ_CLKS_ON; trace_ufshcd_clk_gating(dev_name(hba->dev), hba->clk_gating.state); - if (queue_work(hba->clk_gating.clk_gating_workq, - &hba->clk_gating.ungate_work)) - ufshcd_scsi_block_requests(hba); + queue_work(hba->clk_gating.clk_gating_workq, + &hba->clk_gating.ungate_work); /* * fall through to check if we should wait for this * work to be done or not. */ fallthrough; case REQ_CLKS_ON: - if (async) { - rc = -EAGAIN; - hba->clk_gating.active_reqs--; - break; - } - spin_unlock_irqrestore(hba->host->host_lock, flags); flush_work(&hba->clk_gating.ungate_work); /* Make sure state is CLKS_ON before returning */ @@ -1837,8 +1821,6 @@ int ufshcd_hold(struct ufs_hba *hba, bool async) break; } spin_unlock_irqrestore(hba->host->host_lock, flags); -out: - return rc; } EXPORT_SYMBOL_GPL(ufshcd_hold); @@ -2070,7 +2052,7 @@ static void ufshcd_exit_clk_gating(struct ufs_hba *hba) ufshcd_remove_clk_gating_sysfs(hba); /* Ungate the clock if necessary. */ - ufshcd_hold(hba, false); + ufshcd_hold(hba); hba->clk_gating.is_initialized = false; ufshcd_release(hba); @@ -2466,7 +2448,7 @@ int ufshcd_send_uic_cmd(struct ufs_hba *hba, struct uic_command *uic_cmd) if (hba->quirks & UFSHCD_QUIRK_BROKEN_UIC_CMD) return 0; - ufshcd_hold(hba, false); + ufshcd_hold(hba); mutex_lock(&hba->uic_cmd_mutex); ufshcd_add_delay_before_dme_cmd(hba); @@ -2868,12 +2850,6 @@ static int ufshcd_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *cmd) WARN_ONCE(tag < 0 || tag >= hba->nutrs, "Invalid tag %d\n", tag); - /* - * Allows the UFS error handler to wait for prior ufshcd_queuecommand() - * calls. - */ - rcu_read_lock(); - switch (hba->ufshcd_state) { case UFSHCD_STATE_OPERATIONAL: break; @@ -2919,13 +2895,7 @@ static int ufshcd_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *cmd) hba->req_abort_count = 0; - err = ufshcd_hold(hba, true); - if (err) { - err = SCSI_MLQUEUE_HOST_BUSY; - goto out; - } - WARN_ON(ufshcd_is_clkgating_allowed(hba) && - (hba->clk_gating.state != CLKS_ON)); + ufshcd_hold(hba); lrbp = &hba->lrb[tag]; lrbp->cmd = cmd; @@ -2953,8 +2923,6 @@ static int ufshcd_queuecommand(struct Scsi_Host *host, struct scsi_cmnd *cmd) ufshcd_send_command(hba, tag, hwq); out: - rcu_read_unlock(); - if (ufs_trigger_eh()) { unsigned long flags; @@ -3248,7 +3216,7 @@ int ufshcd_query_flag(struct ufs_hba *hba, enum query_opcode opcode, BUG_ON(!hba); - ufshcd_hold(hba, false); + ufshcd_hold(hba); mutex_lock(&hba->dev_cmd.lock); ufshcd_init_query(hba, &request, &response, opcode, idn, index, selector); @@ -3322,7 +3290,7 @@ int ufshcd_query_attr(struct ufs_hba *hba, enum query_opcode opcode, return -EINVAL; } - ufshcd_hold(hba, false); + ufshcd_hold(hba); mutex_lock(&hba->dev_cmd.lock); ufshcd_init_query(hba, &request, &response, opcode, idn, index, @@ -3418,7 +3386,7 @@ static int __ufshcd_query_descriptor(struct ufs_hba *hba, return -EINVAL; } - ufshcd_hold(hba, false); + ufshcd_hold(hba); mutex_lock(&hba->dev_cmd.lock); ufshcd_init_query(hba, &request, &response, opcode, idn, index, @@ -4236,7 +4204,7 @@ int ufshcd_uic_change_pwr_mode(struct ufs_hba *hba, u8 mode) uic_cmd.command = UIC_CMD_DME_SET; uic_cmd.argument1 = UIC_ARG_MIB(PA_PWRMODE); uic_cmd.argument3 = mode; - ufshcd_hold(hba, false); + ufshcd_hold(hba); ret = ufshcd_uic_pwr_ctrl(hba, &uic_cmd); ufshcd_release(hba); @@ -4343,7 +4311,7 @@ void ufshcd_auto_hibern8_update(struct ufs_hba *hba, u32 ahit) if (update && !pm_runtime_suspended(&hba->ufs_device_wlun->sdev_gendev)) { ufshcd_rpm_get_sync(hba); - ufshcd_hold(hba, false); + ufshcd_hold(hba); ufshcd_auto_hibern8_enable(hba); ufshcd_release(hba); ufshcd_rpm_put_sync(hba); @@ -4936,7 +4904,7 @@ static int ufshcd_verify_dev_init(struct ufs_hba *hba) int err = 0; int retries; - ufshcd_hold(hba, false); + ufshcd_hold(hba); mutex_lock(&hba->dev_cmd.lock); for (retries = NOP_OUT_RETRIES; retries > 0; retries--) { err = ufshcd_exec_dev_cmd(hba, DEV_CMD_TYPE_NOP, @@ -6221,14 +6189,14 @@ static void ufshcd_err_handling_prepare(struct ufs_hba *hba) ufshcd_setup_vreg(hba, true); ufshcd_config_vreg_hpm(hba, hba->vreg_info.vccq); ufshcd_config_vreg_hpm(hba, hba->vreg_info.vccq2); - ufshcd_hold(hba, false); + ufshcd_hold(hba); if (!ufshcd_is_clkgating_allowed(hba)) ufshcd_setup_clocks(hba, true); ufshcd_release(hba); pm_op = hba->is_sys_suspended ? UFS_SYSTEM_PM : UFS_RUNTIME_PM; ufshcd_vops_resume(hba, pm_op); } else { - ufshcd_hold(hba, false); + ufshcd_hold(hba); if (ufshcd_is_clkscaling_supported(hba) && hba->clk_scaling.is_enabled) ufshcd_suspend_clkscaling(hba); @@ -6236,7 +6204,7 @@ static void ufshcd_err_handling_prepare(struct ufs_hba *hba) } ufshcd_scsi_block_requests(hba); /* Drain ufshcd_queuecommand() */ - synchronize_rcu(); + blk_mq_wait_quiesce_done(&hba->host->tag_set); cancel_work_sync(&hba->eeh_work); } @@ -6881,7 +6849,7 @@ static int __ufshcd_issue_tm_cmd(struct ufs_hba *hba, return PTR_ERR(req); req->end_io_data = &wait; - ufshcd_hold(hba, false); + ufshcd_hold(hba); spin_lock_irqsave(host->host_lock, flags); @@ -7117,7 +7085,7 @@ int ufshcd_exec_raw_upiu_cmd(struct ufs_hba *hba, cmd_type = DEV_CMD_TYPE_NOP; fallthrough; case UPIU_TRANSACTION_QUERY_REQ: - ufshcd_hold(hba, false); + ufshcd_hold(hba); mutex_lock(&hba->dev_cmd.lock); err = ufshcd_issue_devman_upiu_cmd(hba, req_upiu, rsp_upiu, desc_buff, buff_len, @@ -7183,7 +7151,7 @@ int ufshcd_advanced_rpmb_req_handler(struct ufs_hba *hba, struct utp_upiu_req *r u16 ehs_len; /* Protects use of hba->reserved_slot. */ - ufshcd_hold(hba, false); + ufshcd_hold(hba); mutex_lock(&hba->dev_cmd.lock); down_read(&hba->clk_scaling_lock); @@ -7417,7 +7385,7 @@ static int ufshcd_abort(struct scsi_cmnd *cmd) WARN_ONCE(tag < 0, "Invalid tag %d\n", tag); - ufshcd_hold(hba, false); + ufshcd_hold(hba); reg = ufshcd_readl(hba, REG_UTP_TRANSFER_REQ_DOOR_BELL); /* If command is already aborted/completed, return FAILED. */ if (!(test_bit(tag, &hba->outstanding_reqs))) { @@ -9410,7 +9378,7 @@ static int __ufshcd_wl_suspend(struct ufs_hba *hba, enum ufs_pm_op pm_op) * If we can't transition into any of the low power modes * just gate the clocks. */ - ufshcd_hold(hba, false); + ufshcd_hold(hba); hba->clk_gating.is_suspended = true; if (ufshcd_is_clkscaling_supported(hba)) diff --git a/include/ufs/ufshcd.h b/include/ufs/ufshcd.h index db2e669985d5..40c537a3880b 100644 --- a/include/ufs/ufshcd.h +++ b/include/ufs/ufshcd.h @@ -1357,7 +1357,7 @@ void ufshcd_fixup_dev_quirks(struct ufs_hba *hba, int ufshcd_read_string_desc(struct ufs_hba *hba, u8 desc_index, u8 **buf, bool ascii); -int ufshcd_hold(struct ufs_hba *hba, bool async); +void ufshcd_hold(struct ufs_hba *hba); void ufshcd_release(struct ufs_hba *hba); void ufshcd_clkgate_delay_set(struct device *dev, unsigned long value);