From patchwork Tue Jul 2 21:16:33 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 168380 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4740949ilk; Tue, 2 Jul 2019 14:16:57 -0700 (PDT) X-Google-Smtp-Source: APXvYqx2AoKIe0OPPd0oN11/14oyayfAPo3rzFKc1xlZkNibEE1zlarx6a54lXt5diXAfNHoJ9qc X-Received: by 2002:adf:80e4:: with SMTP id 91mr17826813wrl.246.1562102217355; Tue, 02 Jul 2019 14:16:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1562102217; cv=none; d=google.com; s=arc-20160816; b=1IcWYIB9hhTBT1Zv2CXpgYYlmvWTDYMBdu/teWX9TiMGaZeo5Rrb4zpa26x/XveRpE 4cdhky8Oe3TV1bApPZUGM6kLC/OxvhUkEZevZ3WYFp30Ys2u4J/4nYUtlJkCK6PM/Scn pryJXp7Mb9CbKWa23+29+M0zndM4KrK3pLmQirTslfsVcoVYRBd2k1xMuYEWFKKhWc8I f3JKtmT8SnCoVYlNUwaMKRe4R2OO/N9so4f12T79geuxROQ0gdGK266ja9S/I2c/mks7 EzzFtdnuxgpzCnaSrtbivANPegXiQP+BEt4utiHbweLXDKd5pnpq4/DkZkSTc90j0v3t bpFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:cc:to:from; bh=XpMx2ueP7X5+cBg0Bd8ZFKl8gSFMnlc5Um0PhDvxc64=; b=yC95R206zxTK4CkrhU3dX6vmqLL74qkq14zsVJ0s+Yf1qn/5qnCg3EmPAqF+h64ZLr URlYw3i0XVnaxyXFJK4iwZl9dFiUFEANnsaPLbCG9cVEH/JoNKUPwavT9LSGiQoEelYa d6eiyxEQg6nvpfr7kxal/SdxnVX+UbeBTGfg1PJ/M2Fn6cCL6tS4Ca7PpC064WRNr6/l jSkBnjKalmOeoT/lWUA8FQdbYDCPG4EgwJk8Z4AHVxEnzAmm+oaaxDp1zsqw7bemtaCd LVqMj1M+CtfRY7D9BFKqhF4I6hQhQl1vqO+9azyxwLOx7EoLBZ7fykCFLSlhA2wOP/mT g+Tg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id v188si66721wmg.155.2019.07.02.14.16.57; Tue, 02 Jul 2019 14:16:57 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id DDDA61BDF3; Tue, 2 Jul 2019 23:16:51 +0200 (CEST) Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by dpdk.org (Postfix) with ESMTP id 3FA781BDEE; Tue, 2 Jul 2019 23:16:50 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9406CCFC; Tue, 2 Jul 2019 14:16:49 -0700 (PDT) Received: from qc2400f-1.austin.arm.com (qc2400f-1.austin.arm.com [10.118.12.65]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 8092C3F703; Tue, 2 Jul 2019 14:16:49 -0700 (PDT) From: Honnappa Nagarahalli To: yipeng1.wang@intel.com, sameh.gobriel@intel.com, bruce.richardson@intel.com, pablo.de.lara.guarch@intel.com, honnappa.nagarahalli@arm.com Cc: gavin.hu@arm.com, ruifeng.wang@arm.com, dev@dpdk.org, nd@arm.com, stable@dpdk.org Date: Tue, 2 Jul 2019 16:16:33 -0500 Message-Id: <20190702211634.37940-2-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190702211634.37940-1-honnappa.nagarahalli@arm.com> References: <20190625211520.43181-1-honnappa.nagarahalli@arm.com> <20190702211634.37940-1-honnappa.nagarahalli@arm.com> Subject: [dpdk-dev] [PATCH v2 1/2] lib/hash: use ordered loads only if signature matches X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" Relaxed signature comparison is done first. Further ordered loads are done only if the signature matches. Any false positives are caught by the full key comparison. This provides performance benefits as load-acquire is executed only when required. Fixes: e605a1d36 ("hash: add lock-free r/w concurrency") Cc: stable@dpdk.org Signed-off-by: Honnappa Nagarahalli Reviewed-by: Gavin Hu Tested-by: Ruifeng Wang Acked-by: Yipeng Wang --- lib/librte_hash/rte_cuckoo_hash.c | 37 +++++++++++++++++++------------ 1 file changed, 23 insertions(+), 14 deletions(-) -- 2.17.1 diff --git a/lib/librte_hash/rte_cuckoo_hash.c b/lib/librte_hash/rte_cuckoo_hash.c index 953928f27..0e042d924 100644 --- a/lib/librte_hash/rte_cuckoo_hash.c +++ b/lib/librte_hash/rte_cuckoo_hash.c @@ -1188,22 +1188,31 @@ search_one_bucket_lf(const struct rte_hash *h, const void *key, uint16_t sig, struct rte_hash_key *k, *keys = h->key_store; for (i = 0; i < RTE_HASH_BUCKET_ENTRIES; i++) { - key_idx = __atomic_load_n(&bkt->key_idx[i], + /* Signature comparison is done before the acquire-load + * of the key index to achieve better performance. + * This can result in the reader loading old signature + * (which matches), while the key_idx is updated to a + * value that belongs to a new key. However, the full + * key comparison will ensure that the lookup fails. + */ + if (bkt->sig_current[i] == sig) { + key_idx = __atomic_load_n(&bkt->key_idx[i], __ATOMIC_ACQUIRE); - if (bkt->sig_current[i] == sig && key_idx != EMPTY_SLOT) { - k = (struct rte_hash_key *) ((char *)keys + - key_idx * h->key_entry_size); - pdata = __atomic_load_n(&k->pdata, - __ATOMIC_ACQUIRE); + if (key_idx != EMPTY_SLOT) { + k = (struct rte_hash_key *) ((char *)keys + + key_idx * h->key_entry_size); + pdata = __atomic_load_n(&k->pdata, + __ATOMIC_ACQUIRE); - if (rte_hash_cmp_eq(key, k->key, h) == 0) { - if (data != NULL) - *data = pdata; - /* - * Return index where key is stored, - * subtracting the first dummy index - */ - return key_idx - 1; + if (rte_hash_cmp_eq(key, k->key, h) == 0) { + if (data != NULL) + *data = pdata; + /* + * Return index where key is stored, + * subtracting the first dummy index + */ + return key_idx - 1; + } } } }