From patchwork Wed Oct 24 01:32:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Honnappa Nagarahalli X-Patchwork-Id: 149471 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp1400548ljp; Tue, 23 Oct 2018 18:32:41 -0700 (PDT) X-Google-Smtp-Source: AJdET5eWKio2I9xckfjjBsAIJ2C65VT5timEwtcV5QzHD9ic6AGt1n8ynZP74ugkPGf+U8mMpXp2 X-Received: by 2002:adf:9441:: with SMTP id 59-v6mr518210wrq.305.1540344761756; Tue, 23 Oct 2018 18:32:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1540344761; cv=none; d=google.com; s=arc-20160816; b=iFerqWUQSjK3n39CwmcPG4CHujJZ13JeXihuPZasMZ1QSa6t4UL9pjbK+PmPEf+bHq wWg3BCU7wVd8fVpF6INOZWSWWZeYGNgfcduIeAfggVg75Q0kVdw0QA+mjOiKMgpyUFJZ A2Jd647FxOZIxFFQSYEc5D6ABxZrjHhsk1xP630M7nHGzUdaF59cBqjgQ5F8b+5THMGg 1mOnXy/VS+GmfV2+MF6RC6sb86FvNvPwvBTdZemwkmVul9Q42KY+k1Y8uC2jYhuHnPx8 Uc54DYcd+LD5dN/4fR9ZMeKdphJ0icPqhFC24OJfEplnzerOaB0bFHAfXrDMkk7qHEAg q++A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:message-id:date:cc:to :from; bh=P22isIL8KvizF+t0I0SEXg2rILq4CVIOO+zeQpizi0w=; b=LqT+GtQzyr8Q5/M8RHh+8eVcTkVHu3PO13oD4RxnVDDOLWmM+M1Axbg6sLuMwVScMJ u+MjCktLTCVgx0uN5XEGXWY1pBFLOIRDuJEMV0Fk4VRxTzfsTa28y1mPA0KVmDOOExa6 f3rxSVb/XLhlrl7MBpQoK0ZbxZXsDTbxnY9Il9QiyEnHLj686Y3VWxryLyR6n4+koAlp 9kJojr36iGm5n/oM3O0v965wqSqqn3tHv4Q3yQSImCwZWBs0HTcBb4trh3TC6DLJMGBF ZAYmuqpRcck+8aS2dOECR5BLCUJ9JXwxIB9pa/KSsX1D6Y1sP4yQnl+SB0pMNkE1zsO4 YUIg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Return-Path: Received: from dpdk.org (dpdk.org. [92.243.14.124]) by mx.google.com with ESMTP id c141-v6si2327149wmd.180.2018.10.23.18.32.41; Tue, 23 Oct 2018 18:32:41 -0700 (PDT) Received-SPF: pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) client-ip=92.243.14.124; Authentication-Results: mx.google.com; spf=pass (google.com: domain of dev-bounces@dpdk.org designates 92.243.14.124 as permitted sender) smtp.mailfrom=dev-bounces@dpdk.org Received: from [92.243.14.124] (localhost [127.0.0.1]) by dpdk.org (Postfix) with ESMTP id C95D55F2E; Wed, 24 Oct 2018 03:32:39 +0200 (CEST) Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com [217.140.101.70]) by dpdk.org (Postfix) with ESMTP id BEE295B2A for ; Wed, 24 Oct 2018 03:32:38 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id AF6C2341; Tue, 23 Oct 2018 18:32:37 -0700 (PDT) Received: from 2p2660v4-1.austin.arm.com (2p2660v4-1.austin.arm.com [10.118.14.139]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4885D3F627; Tue, 23 Oct 2018 18:32:37 -0700 (PDT) From: Honnappa Nagarahalli To: bruce.richardson@intel.com, pablo.de.lara.guarch@intel.com Cc: dev@dpdk.org, yipeng1.wang@intel.com, honnappa.nagarahalli@arm.com, dharmik.thakkar@arm.com, gavin.hu@arm.com, nd@arm.com Date: Tue, 23 Oct 2018 20:32:21 -0500 Message-Id: <1540344746-29045-1-git-send-email-honnappa.nagarahalli@arm.com> X-Mailer: git-send-email 2.7.4 Subject: [dpdk-dev] [PATCH v6 0/5] Address reader-writer concurrency in rte_hash X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" This patch has dependency on the following patches in the order: http://patchwork.dpdk.org/cover/45611/ http://patchwork.dpdk.org/patch/47196/ Currently, reader-writer concurrency problems in rte_hash are addressed using reader-writer locks. Use of reader-writer locks results in following issues: 1) In many of the use cases for the hash table, writer threads are running on control plane. If the writer is preempted while holding the lock, it will block the readers for an extended period resulting in packet drops. This problem seems to apply for platforms with transactional memory support as well because of the algorithm used for rte_rwlock_write_lock_tm: static inline void rte_rwlock_write_lock_tm(rte_rwlock_t *rwl) { if (likely(rte_try_tm(&rwl->cnt))) return; rte_rwlock_write_lock(rwl); } i.e. there is a posibility of using rte_rwlock_write_lock in failure cases. 2) Reader-writer lock based solution does not address the following issue. rte_hash_lookup_xxx APIs return the index of the element in the key store. Application(reader) can use that index to reference other data structures in its scope. Because of this, the index should not be freed till the application completes using the index. 3) Since writer blocks all the readers, the hash lookup rate comes down significantly when there is activity on the writer. This happens even for unrelated entries. Performance numbers given below clearly indicate this. Lock-free solution is required to solve these problems. This patch series adds the lock-free capabilities in the following steps: 1) Add support to not free the key-store index upon calling rte_hash_del_xxx APIs. This solves the issue in 2). 2) Correct the alignment for the key store entry to prep for memory ordering. 3) Add memory ordering to prevent race conditions when a new key is added to the table. 4) Reader-writer concurrency issue, caused by moving the keys to their alternate locations during key insert, is solved by introducing an atomic global counter indicating a change in table. 5) This solution also has to solve the issue of readers using key store element even after the key is deleted from control plane. To solve this issue, the hash_del_key_xxx APIs do not free the key store element when lock-free algorithm is enabled. The key store element has to be freed using the newly introduced rte_hash_free_key_with_position API. It needs to be called once all the readers have stopped using the key store element. How this is determined is outside the scope of this patch (RCU is one such mechanism that the application can use). 6) Finally, a lock free reader-writer concurrency flag is added to enable this feature at run time. Performance numbers can be got from the additional test case added as part of this patch. v5->v6 1) Added lock-free RW concurrency tests to autotest_data.py and meson test 2) Fixed Yipeng's email ID 3) Added Bruce's Acked-by v4->v5 1) Rebased with patch v8 of extendable hash bucket feature (http://patchwork.dpdk.org/patch/47196/) 2) Changed 'success' to 'hit' and 'fail' to 'miss' in read/write concurrenct lock free test code v3-v4 1) Merged 4/7, 5/7 and 6/7 into 4/5 2) Changed RTE_HASH_EXTRA_FLAGS_RECYCLE_ON_DEL to RTE_HASH_EXTRA_FLAGS_NO_FREE_ON_DEL (Yipeng) 3) Changed the commit log for the patch "hash: correct key store element alignment" (Yipeng) 4) Changed the comment for rte_hash_add_key_data API (Yipeng) 5) Added bulk lookup for lock-free performance test case (Yipeng) 5) Reduced the number of keys to 4M in the tests (Yipeng) v2->v3 1) Rebased on top of: http://patchwork.dpdk.org/cover/45611/ http://patchwork.dpdk.org/project/dpdk/list/?series=1822 2) Added comments to RTE_HASH_EXTRA_FLAGS_RW_CONCURRENCY_LF to indicate multi writer support (Yipeng) 3) Updated the comments for rte_hash_add_key_data_xxx APIs to free the 'data' only after the readers have completed using 'data' (Yipeng) 4) Extendable tables are not supported when lock free algorithm is requested. v1->v2 1) Separate multi-writer capability from rw concurrency 2) Add do not recycle on delete feature (Yipeng) 3) Add Arm copyright 4) Add test case to test lock-free algorithm and multi-writer test case (Yipeng) 5) Additional API documentation to indicate RCU usage (Yipeng) 6) Additional documentation on rte_hash_reset API (Yipeng) 7) Allocate memory for the global counter and avoid API changes (Yipeng) Dharmik Thakkar (1): test/hash: read-write lock-free concurrency test Honnappa Nagarahalli (4): hash: separate multi-writer from rw-concurrency hash: support do not free on delete hash: fix key store element alignment hash: add lock-free read-write concurrency lib/librte_hash/rte_cuckoo_hash.c | 520 +++++++++++---- lib/librte_hash/rte_cuckoo_hash.h | 21 +- lib/librte_hash/rte_hash.h | 77 ++- lib/librte_hash/rte_hash_version.map | 7 + test/test/Makefile | 1 + test/test/autotest_data.py | 6 + test/test/meson.build | 2 + test/test/test_hash.c | 140 +++- test/test/test_hash_readwrite.c | 6 +- test/test/test_hash_readwrite_lf.c | 1220 ++++++++++++++++++++++++++++++++++ 10 files changed, 1855 insertions(+), 145 deletions(-) create mode 100644 test/test/test_hash_readwrite_lf.c -- 2.7.4