From patchwork Tue Aug 1 15:00:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 109173 Delivered-To: patch@linaro.org Received: by 10.140.101.6 with SMTP id t6csp1604572qge; Tue, 1 Aug 2017 08:01:20 -0700 (PDT) X-Received: by 10.200.39.212 with SMTP id x20mr28410630qtx.157.1501599680348; Tue, 01 Aug 2017 08:01:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501599680; cv=none; d=google.com; s=arc-20160816; b=QGlRE9Etaa6wBVP7TYbTzMcT8oCE7A0darkLY3Hsrr0zOnCFz71r54I9HhPT3F/Ugi y8MZS2BJZ7cYWAuImrpTG3hSXrSya+NEUytQ5PsHvB6OYvP1xEgDYOkIyeaXaHUylJ7C ZoylTqlxWoMwGAmUZO0gbB0ksnlhuRp2R/1hjFHVhXKDKzxoLFddnxiIcnTqkxLp3E5w p/L0nLk/PrBTD/ldSn/NxE7XIW2eqv1bNBH1zbQp69KHHghjXpSHTGoJ8Iz6MAwOfgtZ 7KDAsFW8Dw91zjbxRp9NQUGBseuj9ny2YGWzolRyTREoIm1WTnzVNrXqSv/rBw3VwAnE mhTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=1pQsV9MmEfotzut7N1Ke2PeKv1WmpaCIbaeQueon9Wg=; b=ZWs6BrT6wycEzRq4stnBTDKKwHrnfIx1nBCUerfrOPAAesDvlJZ9bGcxMWpMLoD6U2 Oerif4EOZx0vvDy/su2vVcsUp/9fzu/P9Z8R978Vg+f2VN7GuXaRU1hey0XjTWgUrysx YgRwxVz02aoN6Ig8k9RBir8Lxpfn+ADnieNh/VykjmxeBxUhvk+/9Eku0LTPBJAw6M00 UPJEvTaztsDGVU8NiQ0GmofbeTwyOi05sRnqxbF3FOzIWrnyYDFYHYytMUuO6eBQUS25 SDHSJr/fUYtME06Xxa55HnR2ufKFDfcIOpzDPIn0f5LkG23uoY3GxO8iQ8CTS/y7qN48 b+cg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id r63si25834928qki.145.2017.08.01.08.01.20; Tue, 01 Aug 2017 08:01:20 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 08759644E6; Tue, 1 Aug 2017 15:01:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 731D2644FC; Tue, 1 Aug 2017 15:00:38 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 871A86099C; Tue, 1 Aug 2017 15:00:34 +0000 (UTC) Received: from forward1o.cmail.yandex.net (forward1o.cmail.yandex.net [37.9.109.84]) by lists.linaro.org (Postfix) with ESMTPS id 558276099C for ; Tue, 1 Aug 2017 15:00:27 +0000 (UTC) Received: from smtp2j.mail.yandex.net (smtp2j.mail.yandex.net [95.108.130.60]) by forward1o.cmail.yandex.net (Yandex) with ESMTP id C1FB72101F for ; Tue, 1 Aug 2017 18:00:25 +0300 (MSK) Received: from smtp2j.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp2j.mail.yandex.net (Yandex) with ESMTP id BC0CB3EC0A70 for ; Tue, 1 Aug 2017 18:00:24 +0300 (MSK) Received: by smtp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id tyzQTD0Qgu-0NbiKvBA; Tue, 01 Aug 2017 18:00:23 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) X-Yandex-Suid-Status: 1 0 From: Github ODP bot To: lng-odp@lists.linaro.org Date: Tue, 1 Aug 2017 18:00:01 +0300 Message-Id: <1501599610-28388-2-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> References: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 90 Subject: [lng-odp] [PATCH v3 1/10] linux-generic: packet: restructure inline routines to use macros X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Bill Fischofer Use an internal macro for accessing inline fields in preparation for introducing packet reference support. Signed-off-by: Bill Fischofer --- /** Email created from pull request 90 (Bill-Fischofer-Linaro:pktrefs) ** https://github.com/Linaro/odp/pull/90 ** Patch: https://github.com/Linaro/odp/pull/90.patch ** Base sha: bb0ca193f8c28036fdf8b3d6c8aa5d4f11980e6c ** Merge commit sha: dfef320840f96825686124135a6e509db80b7b82 **/ .../include/odp/api/plat/packet_flag_inlines.h | 3 +- .../include/odp/api/plat/packet_inlines.h | 40 ++++++++-------------- .../include/odp/api/plat/packet_types.h | 4 +++ 3 files changed, 19 insertions(+), 28 deletions(-) diff --git a/platform/linux-generic/include/odp/api/plat/packet_flag_inlines.h b/platform/linux-generic/include/odp/api/plat/packet_flag_inlines.h index 2de04c81..42003e31 100644 --- a/platform/linux-generic/include/odp/api/plat/packet_flag_inlines.h +++ b/platform/linux-generic/include/odp/api/plat/packet_flag_inlines.h @@ -22,8 +22,7 @@ extern const _odp_packet_inline_offset_t _odp_packet_inline; /** @internal Inline function @param pkt @return */ static inline uint64_t _odp_packet_input_flags(odp_packet_t pkt) { - return *(uint64_t *)(uintptr_t)((uint8_t *)pkt + - _odp_packet_inline.input_flags); + return _odp_pkt_get(pkt, uint64_t, input_flags); } /** @internal Inline function @param pkt @return */ diff --git a/platform/linux-generic/include/odp/api/plat/packet_inlines.h b/platform/linux-generic/include/odp/api/plat/packet_inlines.h index 3dd643fe..d0cf1390 100644 --- a/platform/linux-generic/include/odp/api/plat/packet_inlines.h +++ b/platform/linux-generic/include/odp/api/plat/packet_inlines.h @@ -38,91 +38,79 @@ static inline odp_packet_seg_t _odp_packet_seg_from_ndx(uint32_t ndx) /** @internal Inline function @param pkt @return */ static inline void *_odp_packet_data(odp_packet_t pkt) { - return *(void **)(uintptr_t)((uint8_t *)pkt + _odp_packet_inline.data); + return _odp_pkt_get(pkt, void *, data); } /** @internal Inline function @param pkt @return */ static inline uint32_t _odp_packet_seg_len(odp_packet_t pkt) { - return *(uint32_t *)(uintptr_t)((uint8_t *)pkt + - _odp_packet_inline.seg_len); + return _odp_pkt_get(pkt, uint32_t, seg_len); } /** @internal Inline function @param pkt @return */ static inline uint32_t _odp_packet_len(odp_packet_t pkt) { - return *(uint32_t *)(uintptr_t)((uint8_t *)pkt + - _odp_packet_inline.frame_len); + return _odp_pkt_get(pkt, uint32_t, frame_len); } /** @internal Inline function @param pkt @return */ static inline uint32_t _odp_packet_headroom(odp_packet_t pkt) { - return *(uint32_t *)(uintptr_t)((uint8_t *)pkt + - _odp_packet_inline.headroom); + return _odp_pkt_get(pkt, uint32_t, headroom); } /** @internal Inline function @param pkt @return */ static inline uint32_t _odp_packet_tailroom(odp_packet_t pkt) { - return *(uint32_t *)(uintptr_t)((uint8_t *)pkt + - _odp_packet_inline.tailroom); + return _odp_pkt_get(pkt, uint32_t, tailroom); } /** @internal Inline function @param pkt @return */ static inline odp_pool_t _odp_packet_pool(odp_packet_t pkt) { - return *(odp_pool_t *)(uintptr_t)((uint8_t *)pkt + - _odp_packet_inline.pool); + return _odp_pkt_get(pkt, odp_pool_t, pool); } /** @internal Inline function @param pkt @return */ static inline odp_pktio_t _odp_packet_input(odp_packet_t pkt) { - return *(odp_pktio_t *)(uintptr_t)((uint8_t *)pkt + - _odp_packet_inline.input); + return _odp_pkt_get(pkt, odp_pktio_t, input); } /** @internal Inline function @param pkt @return */ static inline int _odp_packet_num_segs(odp_packet_t pkt) { - return *(uint8_t *)(uintptr_t)((uint8_t *)pkt + - _odp_packet_inline.segcount); + return _odp_pkt_get(pkt, uint8_t, segcount); } /** @internal Inline function @param pkt @return */ static inline void *_odp_packet_user_ptr(odp_packet_t pkt) { - return *(void **)(uintptr_t)((uint8_t *)pkt + - _odp_packet_inline.user_ptr); + return _odp_pkt_get(pkt, void *, user_ptr); } /** @internal Inline function @param pkt @return */ static inline void *_odp_packet_user_area(odp_packet_t pkt) { - return *(void **)(uintptr_t)((uint8_t *)pkt + - _odp_packet_inline.user_area); + return _odp_pkt_get(pkt, void *, user_area); } /** @internal Inline function @param pkt @return */ static inline uint32_t _odp_packet_user_area_size(odp_packet_t pkt) { - return *(uint32_t *)(uintptr_t)((uint8_t *)pkt + - _odp_packet_inline.user_area_size); + return _odp_pkt_get(pkt, uint32_t, user_area_size); } /** @internal Inline function @param pkt @return */ static inline uint32_t _odp_packet_flow_hash(odp_packet_t pkt) { - return *(uint32_t *)(uintptr_t)((uint8_t *)pkt + - _odp_packet_inline.flow_hash); + return _odp_pkt_get(pkt, uint32_t, flow_hash); } /** @internal Inline function @param pkt @return */ static inline odp_time_t _odp_packet_ts(odp_packet_t pkt) { - return *(odp_time_t *)(uintptr_t)((uint8_t *)pkt + - _odp_packet_inline.timestamp); + return _odp_pkt_get(pkt, odp_time_t, timestamp); } /** @internal Inline function @param pkt @return */ @@ -134,7 +122,7 @@ static inline void *_odp_packet_head(odp_packet_t pkt) /** @internal Inline function @param pkt @return */ static inline int _odp_packet_is_segmented(odp_packet_t pkt) { - return _odp_packet_num_segs(pkt) > 1; + return _odp_pkt_get(pkt, uint8_t, segcount) > 1; } /** @internal Inline function @param pkt @return */ diff --git a/platform/linux-generic/include/odp/api/plat/packet_types.h b/platform/linux-generic/include/odp/api/plat/packet_types.h index a209c759..41468bec 100644 --- a/platform/linux-generic/include/odp/api/plat/packet_types.h +++ b/platform/linux-generic/include/odp/api/plat/packet_types.h @@ -67,6 +67,10 @@ typedef enum { #endif +/** @internal Packet field accessor */ +#define _odp_pkt_get(pkt, cast, field) \ + (*(cast *)(uintptr_t)((uint8_t *)pkt + _odp_packet_inline.field)) + /** @internal Packet header field offsets for inline functions */ typedef struct _odp_packet_inline_offset_t { /** @internal field offset */ From patchwork Tue Aug 1 15:00:02 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 109176 Delivered-To: patch@linaro.org Received: by 10.140.101.6 with SMTP id t6csp1609363qge; Tue, 1 Aug 2017 08:04:36 -0700 (PDT) X-Received: by 10.55.209.86 with SMTP id s83mr25004536qki.171.1501599876405; Tue, 01 Aug 2017 08:04:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501599876; cv=none; d=google.com; s=arc-20160816; b=RswSVA+ugm4A53wU9TIfWPkq+zcQzH8YnhIbNIee6uo8DiC1pf6qPJm+qXUPubkZf7 qo8ixUQZjFQQDsAMymjvZwtjoAMHsIGW2UqshLleWRXqinFsU98cl4jZvbEqEebnbzcN EOIhxBv+cJavBSFaxX8+X/3EHarakvj6yT3PIDap3DGBpDIzE2oujVGNDpf0n8YMf0NY ManSKSYgc4kaQKMguBW6CxtAjQ+igTVCpgntNiU1+T9X4Yq9fZN8QAubq7qRfkxWnou/ 7vSftWbLMzEhR0XRs/osnLSrZ5uG9iprPNflWeTXBPDe/C8z1Doq1gGioWiDKms9vxHD qcKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=WqE5NFljqd03xbBHMlraQAyX5owVsesV03QUHbv7i14=; b=BJakrA9b6G0Qnu+bFbQgQDt+OhF0hmkOH9KiWzsUz+iM8Bidi3ivZhOQ6oITWwGgXQ B64nLugeTPuylWQzJ+V9r5H3UcCIQfwq9yWHqZMQt5uBnU1XJONhseFsnlL4s4o4Qk9p fPU0bxJoRzcjFkCICFN4bKRgvxTEhjowfqLRPXjtglrRig7AahvdeEUKdF5hrMiEKlVI Hy9tEy2XU+zBBMIrFGu4kCNcCah9yCJrQP/HoqmW4V66ULz8+rrpb5QjxgPIQHBbYp9F QCBWQOgDMSOhx13YfAPzgPQRvSWLOpGHnw5db8zdONOx56fV1AGoTIL/FCe4GJ1mYdr+ jm/g== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id 5si26401362qtu.311.2017.08.01.08.04.35; Tue, 01 Aug 2017 08:04:36 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 3A39364A61; Tue, 1 Aug 2017 15:04:35 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id CA89264FD5; Tue, 1 Aug 2017 15:01:23 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 347146099C; Tue, 1 Aug 2017 15:01:10 +0000 (UTC) Received: from forward4m.cmail.yandex.net (forward4m.cmail.yandex.net [5.255.216.22]) by lists.linaro.org (Postfix) with ESMTPS id 0C7BC60B12 for ; Tue, 1 Aug 2017 15:00:31 +0000 (UTC) Received: from smtp2j.mail.yandex.net (smtp2j.mail.yandex.net [IPv6:2a02:6b8:0:801::ac]) by forward4m.cmail.yandex.net (Yandex) with ESMTP id 2EAA52078E for ; Tue, 1 Aug 2017 18:00:29 +0300 (MSK) Received: from smtp2j.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp2j.mail.yandex.net (Yandex) with ESMTP id 907593EC11C6 for ; Tue, 1 Aug 2017 18:00:25 +0300 (MSK) Received: by smtp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id tyzQTD0Qgu-0PbixV6q; Tue, 01 Aug 2017 18:00:25 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) X-Yandex-Suid-Status: 1 0 From: Github ODP bot To: lng-odp@lists.linaro.org Date: Tue, 1 Aug 2017 18:00:02 +0300 Message-Id: <1501599610-28388-3-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> References: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 90 Subject: [lng-odp] [PATCH v3 2/10] linux-generic: packet: add packet fields needed for references X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Bill Fischofer Add new fields to the odp_packet_hdr_t used for support of zero-copy references. Also added are the offsets needed to access them from inlined functions. Signed-off-by: Bill Fischofer --- /** Email created from pull request 90 (Bill-Fischofer-Linaro:pktrefs) ** https://github.com/Linaro/odp/pull/90 ** Patch: https://github.com/Linaro/odp/pull/90.patch ** Base sha: bb0ca193f8c28036fdf8b3d6c8aa5d4f11980e6c ** Merge commit sha: dfef320840f96825686124135a6e509db80b7b82 **/ .../linux-generic/include/odp/api/plat/packet_types.h | 8 ++++++++ platform/linux-generic/include/odp_packet_internal.h | 15 ++++++++++++++- platform/linux-generic/odp_packet.c | 4 ++++ 3 files changed, 26 insertions(+), 1 deletion(-) diff --git a/platform/linux-generic/include/odp/api/plat/packet_types.h b/platform/linux-generic/include/odp/api/plat/packet_types.h index 41468bec..1554c3af 100644 --- a/platform/linux-generic/include/odp/api/plat/packet_types.h +++ b/platform/linux-generic/include/odp/api/plat/packet_types.h @@ -84,6 +84,14 @@ typedef struct _odp_packet_inline_offset_t { /** @internal field offset */ size_t tailroom; /** @internal field offset */ + size_t unshared_len; + /** @internal field offset */ + size_t ref_hdr; + /** @internal field offset */ + size_t ref_offset; + /** *internal field offset */ + size_t ref_len; + /** @internal field offset */ size_t pool; /** @internal field offset */ size_t input; diff --git a/platform/linux-generic/include/odp_packet_internal.h b/platform/linux-generic/include/odp_packet_internal.h index f77987c8..0542dd10 100644 --- a/platform/linux-generic/include/odp_packet_internal.h +++ b/platform/linux-generic/include/odp_packet_internal.h @@ -106,7 +106,7 @@ typedef struct { * packet_init(). Because of this any new fields added must be reviewed for * initialization requirements. */ -typedef struct { +typedef struct odp_packet_hdr_t { /* common buffer header */ odp_buffer_hdr_t buf_hdr; @@ -122,6 +122,19 @@ typedef struct { uint32_t headroom; uint32_t tailroom; + /* Fields used to support packet references */ + uint32_t unshared_len; + /* Next pkt_hdr in reference chain */ + struct odp_packet_hdr_t *ref_hdr; + /* Offset into next pkt_hdr that ref was created at */ + uint32_t ref_offset; + /* frame_len in next pkt_hdr at time ref was created. This + * allows original offset to be maintained when base pkt len + * is changed */ + uint32_t ref_len; + /* Incremented on refs, decremented on frees. */ + odp_atomic_u32_t ref_count; + /* * Members below are not initialized by packet_init() */ diff --git a/platform/linux-generic/odp_packet.c b/platform/linux-generic/odp_packet.c index dbde7280..af8c442e 100644 --- a/platform/linux-generic/odp_packet.c +++ b/platform/linux-generic/odp_packet.c @@ -33,6 +33,10 @@ const _odp_packet_inline_offset_t _odp_packet_inline ODP_ALIGNED_CACHE = { .frame_len = offsetof(odp_packet_hdr_t, frame_len), .headroom = offsetof(odp_packet_hdr_t, headroom), .tailroom = offsetof(odp_packet_hdr_t, tailroom), + .unshared_len = offsetof(odp_packet_hdr_t, unshared_len), + .ref_hdr = offsetof(odp_packet_hdr_t, ref_hdr), + .ref_offset = offsetof(odp_packet_hdr_t, ref_offset), + .ref_len = offsetof(odp_packet_hdr_t, ref_len), .pool = offsetof(odp_packet_hdr_t, buf_hdr.pool_hdl), .input = offsetof(odp_packet_hdr_t, input), .segcount = offsetof(odp_packet_hdr_t, buf_hdr.segcount), From patchwork Tue Aug 1 15:00:03 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 109177 Delivered-To: patch@linaro.org Received: by 10.140.101.6 with SMTP id t6csp1610288qge; Tue, 1 Aug 2017 08:05:18 -0700 (PDT) X-Received: by 10.200.53.251 with SMTP id l56mr9782114qtb.82.1501599918305; Tue, 01 Aug 2017 08:05:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501599918; cv=none; d=google.com; s=arc-20160816; b=EGj2D4lrOP5p1c9TUvydESlTj1qTyKB+aMgg3cWBVY0kDrSsbbtWy7ALa/XSHgDXaf +p+kIhmT6Dh/nYO6V9aV1PVkiQeHlORjJ8jF0FPkrfE5bZGDwqdYfUzjMy1nLhaGzFzp A/hhIWlk23cY12S4voV6Q348BonQEHJyyKqvPobTmEZ8W1Qj6DrxZcdAtOVzc+To0d0g 84B2xLtSuEOMwK7iVpKL4BWoGcdaQvfmkEqbNgbgUMp8OkzEChmEJ6W28sClcimXQBlT f5kf97h3n5k5KsDChwAOr3NqCIL/6lLwTITId3UojwZUzMs8mw0xuGLlk35sX/Yg6KPK 3bzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=/4/M2/gZXVsl1ZhuQbgYBYg3pYZLGWn2bK1igDYs89g=; b=QtpuP3lPb7aOWOy8QMS4YtvCOX4Ab14eHcjvDn3lIUIMtCq9v39X7AnaVHXgMwa51N sZ6dMrMEkiF4JKdNuEeSAJuNP8pl+7VK/5coB7c0O3bmrO9mTYXRQeLIJcJcLDHcIY8+ Ytmw/8f2qau1365zKi0PeqQdio0oKSOR98FA7gxlTcTaYG6YkBzaT+A26ug0oU9a88PD Sc/SX2J6tMbctFhasjSXDyz6Uml9SNbOidRcXdSDqLEBpRvSp5T3iPOwUF7kajYW8WA/ Wga16/HmVziqwxhYoaKNgVmZh8BTgHC0rmfj7Z3SGAKKNFJ0738DTOa87SIrdvNSLrpE oEDA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id i28si27193920qta.182.2017.08.01.08.05.17; Tue, 01 Aug 2017 08:05:18 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id C570365BE7; Tue, 1 Aug 2017 15:05:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 1D7CF65BC1; Tue, 1 Aug 2017 15:01:28 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 1DC8160C1E; Tue, 1 Aug 2017 15:01:14 +0000 (UTC) Received: from forward1h.cmail.yandex.net (forward1h.cmail.yandex.net [87.250.230.16]) by lists.linaro.org (Postfix) with ESMTPS id E215A644E6 for ; Tue, 1 Aug 2017 15:00:35 +0000 (UTC) Received: from smtp2j.mail.yandex.net (smtp2j.mail.yandex.net [IPv6:2a02:6b8:0:801::ac]) by forward1h.cmail.yandex.net (Yandex) with ESMTP id 8678820FE1 for ; Tue, 1 Aug 2017 18:00:34 +0300 (MSK) Received: from smtp2j.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp2j.mail.yandex.net (Yandex) with ESMTP id CFFF43EC11BF for ; Tue, 1 Aug 2017 18:00:27 +0300 (MSK) Received: by smtp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id tyzQTD0Qgu-0Qb4ocuF; Tue, 01 Aug 2017 18:00:26 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) X-Yandex-Suid-Status: 1 0 From: Github ODP bot To: lng-odp@lists.linaro.org Date: Tue, 1 Aug 2017 18:00:03 +0300 Message-Id: <1501599610-28388-4-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> References: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 90 Subject: [lng-odp] [PATCH v3 3/10] linux-generic: packet: add routines for manipulating reference counts X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Bill Fischofer Signed-off-by: Bill Fischofer --- /** Email created from pull request 90 (Bill-Fischofer-Linaro:pktrefs) ** https://github.com/Linaro/odp/pull/90 ** Patch: https://github.com/Linaro/odp/pull/90.patch ** Base sha: bb0ca193f8c28036fdf8b3d6c8aa5d4f11980e6c ** Merge commit sha: dfef320840f96825686124135a6e509db80b7b82 **/ platform/linux-generic/include/odp_packet_internal.h | 19 +++++++++++++++++++ platform/linux-generic/odp_packet.c | 10 ++++++++++ 2 files changed, 29 insertions(+) diff --git a/platform/linux-generic/include/odp_packet_internal.h b/platform/linux-generic/include/odp_packet_internal.h index 0542dd10..8c83966c 100644 --- a/platform/linux-generic/include/odp_packet_internal.h +++ b/platform/linux-generic/include/odp_packet_internal.h @@ -254,6 +254,25 @@ static inline uint32_t packet_len(odp_packet_hdr_t *pkt_hdr) return pkt_hdr->frame_len; } +static inline uint32_t packet_ref_count(odp_packet_hdr_t *pkt_hdr) +{ + /* Breach the atomic type to do a peek at the ref count. This + * is used to bypass atomic operations if ref_count == 1 for + * performance reasons. + */ + return pkt_hdr->ref_count.v; +} + +static inline void packet_ref_count_set(odp_packet_hdr_t *pkt_hdr, uint32_t n) +{ + /* Only used during init when there are no other possible + * references to this pkt, so avoid the "atomic" overhead by + * a controlled breach of the atomic type here. This saves + * over 10% of the pathlength in routines like packet_alloc(). + */ + pkt_hdr->ref_count.v = n; +} + static inline void packet_set_len(odp_packet_hdr_t *pkt_hdr, uint32_t len) { pkt_hdr->frame_len = len; diff --git a/platform/linux-generic/odp_packet.c b/platform/linux-generic/odp_packet.c index af8c442e..988045e4 100644 --- a/platform/linux-generic/odp_packet.c +++ b/platform/linux-generic/odp_packet.c @@ -61,6 +61,16 @@ static inline odp_buffer_t buffer_handle(odp_packet_hdr_t *pkt_hdr) return (odp_buffer_t)pkt_hdr; } +static inline void packet_ref_inc(odp_packet_hdr_t *pkt_hdr) +{ + odp_atomic_inc_u32(&pkt_hdr->ref_count); +} + +static inline uint32_t packet_ref_dec(odp_packet_hdr_t *pkt_hdr) +{ + return odp_atomic_fetch_dec_u32(&pkt_hdr->ref_count); +} + static inline odp_packet_hdr_t *buf_to_packet_hdr(odp_buffer_t buf) { return (odp_packet_hdr_t *)buf_hdl_to_hdr(buf); From patchwork Tue Aug 1 15:00:04 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 109175 Delivered-To: patch@linaro.org Received: by 10.140.101.6 with SMTP id t6csp1608241qge; Tue, 1 Aug 2017 08:03:50 -0700 (PDT) X-Received: by 10.55.161.68 with SMTP id k65mr27578905qke.57.1501599830840; Tue, 01 Aug 2017 08:03:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501599830; cv=none; d=google.com; s=arc-20160816; b=yOpXSrWXUjbILGITnYWwLQIC3gfFNioPpBpSD0nyZV+7Y8xriDBW/5j2bnAL7gxs9v jhvkQtHr6jMJQ0kQUY6HThck/wTo/LOCE8qnqf5M+uaI2+oWBTzi7l3hes6c6Z565G/E a/vhzBxpP8n5sknoNKruR9MePV2Rt1lMoOPaIC7x2/I3xGTOTJYQvAnalQMLzqCIunn9 Qg21pj0JCi/M422RRL6gDDcgnYEroGJo6XeVR9fHX5WE7ALOe7pDVh1mWSccXwBeCy5V Utq0g0lo31MlTQCMnCbztRQnFgVSdScZOmzQ+OtKbJ03M70F8FD25yfCdQPJBQpsd9/N VKLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=KMTCM8m4CuN3U3jAEhtSFGY4VZ9+NEEmRmN/0ZF2SE8=; b=ZVtC1APd/qsMkSMowxxsJLyTVRo6+sRz//ULjlhhJfDsfOF5N8H7LtgK74H6QrGaNA jPS9HPSgIZJrPNM8QtQEhPrrKx9mj3whKR7UmzONpQTDdpAM54W3u+R2ywssISzC8hka ud7qjJpwuZERUj+a7+YxusiH7+cYq0jM9UpgWaDblbpMSzFx66/LibEPHlzhoPy3LYtx VD1qq1t0VScyRPcZqZ0a82BaDs5p1z+H4PCeTn3Z1V0KeklsBrIouKeZsexdUzmi7nAl Lr92Fn3Z1hMR9u6qhfhVm5iwLwaP6H382dir6ctLpI8sfjFpQ3ws31kIqDlstFxyzgsj HkNg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id g125si13413917qkd.327.2017.08.01.08.03.50; Tue, 01 Aug 2017 08:03:50 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 510E060B4E; Tue, 1 Aug 2017 15:03:50 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id A0707644E6; Tue, 1 Aug 2017 15:01:22 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 1F41F60725; Tue, 1 Aug 2017 15:01:10 +0000 (UTC) Received: from forward5j.cmail.yandex.net (forward5j.cmail.yandex.net [5.255.227.23]) by lists.linaro.org (Postfix) with ESMTPS id 79EFC64464 for ; Tue, 1 Aug 2017 15:00:34 +0000 (UTC) Received: from smtp2j.mail.yandex.net (smtp2j.mail.yandex.net [IPv6:2a02:6b8:0:801::ac]) by forward5j.cmail.yandex.net (Yandex) with ESMTP id 27E5921002 for ; Tue, 1 Aug 2017 18:00:33 +0300 (MSK) Received: from smtp2j.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp2j.mail.yandex.net (Yandex) with ESMTP id 61A0D3EC1197 for ; Tue, 1 Aug 2017 18:00:28 +0300 (MSK) Received: by smtp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id tyzQTD0Qgu-0SbewjZM; Tue, 01 Aug 2017 18:00:28 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) X-Yandex-Suid-Status: 1 0 From: Github ODP bot To: lng-odp@lists.linaro.org Date: Tue, 1 Aug 2017 18:00:04 +0300 Message-Id: <1501599610-28388-5-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> References: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 90 Subject: [lng-odp] [PATCH v3 4/10] linux-generic: packet: add initializations for new hdr fields X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Bill Fischofer Add initializations for new packet hdr fields needed to support packet references. Signed-off-by: Bill Fischofer --- /** Email created from pull request 90 (Bill-Fischofer-Linaro:pktrefs) ** https://github.com/Linaro/odp/pull/90 ** Patch: https://github.com/Linaro/odp/pull/90.patch ** Base sha: bb0ca193f8c28036fdf8b3d6c8aa5d4f11980e6c ** Merge commit sha: dfef320840f96825686124135a6e509db80b7b82 **/ platform/linux-generic/include/odp_packet_internal.h | 4 ++++ platform/linux-generic/odp_packet.c | 5 +++++ 2 files changed, 9 insertions(+) diff --git a/platform/linux-generic/include/odp_packet_internal.h b/platform/linux-generic/include/odp_packet_internal.h index 8c83966c..2207ac58 100644 --- a/platform/linux-generic/include/odp_packet_internal.h +++ b/platform/linux-generic/include/odp_packet_internal.h @@ -222,6 +222,10 @@ static inline void packet_init(odp_packet_hdr_t *pkt_hdr, uint32_t len) CONFIG_PACKET_TAILROOM; pkt_hdr->input = ODP_PKTIO_INVALID; + + /* By default packet has no references */ + pkt_hdr->unshared_len = len; + pkt_hdr->ref_hdr = NULL; } static inline void copy_packet_parser_metadata(odp_packet_hdr_t *src_hdr, diff --git a/platform/linux-generic/odp_packet.c b/platform/linux-generic/odp_packet.c index 988045e4..41262cc2 100644 --- a/platform/linux-generic/odp_packet.c +++ b/platform/linux-generic/odp_packet.c @@ -253,6 +253,9 @@ void packet_parse_reset(odp_packet_hdr_t *pkt_hdr) pkt_hdr->p.l2_offset = 0; pkt_hdr->p.l3_offset = ODP_PACKET_OFFSET_INVALID; pkt_hdr->p.l4_offset = ODP_PACKET_OFFSET_INVALID; + + /* Ensure dummy pkt hdrs used in I/O recv classification are valid */ + pkt_hdr->ref_hdr = NULL; } static inline void init_segments(odp_packet_hdr_t *pkt_hdr[], int num) @@ -265,6 +268,7 @@ static inline void init_segments(odp_packet_hdr_t *pkt_hdr[], int num) hdr->buf_hdr.seg[0].data = hdr->buf_hdr.base_data; hdr->buf_hdr.seg[0].len = BASE_LEN; + packet_ref_count_set(hdr, 1); /* Link segments */ if (CONFIG_PACKET_MAX_SEGS != 1) { @@ -274,6 +278,7 @@ static inline void init_segments(odp_packet_hdr_t *pkt_hdr[], int num) for (i = 1; i < num; i++) { odp_buffer_hdr_t *buf_hdr; + packet_ref_count_set(pkt_hdr[i], 1); buf_hdr = &pkt_hdr[i]->buf_hdr; hdr->buf_hdr.seg[i].hdr = buf_hdr; hdr->buf_hdr.seg[i].data = buf_hdr->base_data; From patchwork Tue Aug 1 15:00:05 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 109174 Delivered-To: patch@linaro.org Received: by 10.140.101.6 with SMTP id t6csp1606947qge; Tue, 1 Aug 2017 08:03:00 -0700 (PDT) X-Received: by 10.237.62.156 with SMTP id n28mr29160093qtf.203.1501599780024; Tue, 01 Aug 2017 08:03:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501599780; cv=none; d=google.com; s=arc-20160816; b=H7jiCosEl8zGEu8dOfYzU4mh0LKMPT4xKAFTtvQ2+mpOp6aQo3gF0j/+rZCa+vvzNx 6Awc5kyKUzAYY5TbP8uMKveUrRH1NeRhLE15QoWO3uOaqvykl7GBpQDvC8fyo2aqZ3lU mAnPiRoaiRqC2ZDKhP8U0B32m+gZcEquxW8sYLM77o+/mEfldwVWoMY1xbLH6dsIJxxj kf16+ynOalR26x7Y9zc36S5WiWAi4QgPLsexyYiBn5lsUi1w50eoDOB2Zg7kS+QZCh4F qok8a/sFid+rpNEV7VWukK7HuHJkH6slplXmGA0ZTXti8FpcMBvUrPk/qw4atkMi+dZp EhjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=4c7V/6rQFffCdpRUgSquoUTZmAKeEJB7F3Vu7jcvV9g=; b=VXbK9miJOwy7xtyemRnblo4S9iR9qVjpitZy6JhZEeSJY6va1Yk+SDG1RZG2T/8xV3 XifiOhz7xMZmlmj7/ZkB6g2mdrf9vZmrn5QfaDMCEf7fGoH7uraSNg1PVPn5AFSJNDLa b3acliMmHbZ3DrlEBHkfVeWBJ5AamC52480Rv8iHGLgkYcv0R2BpWHG0sr9cgFVDpQE3 HNbo0GACi8n5kRd/B86lnr0Y4ioiBu7XJV22HxjrTUiY5bGrkpYgSlND2Xy0p09OIO9r moDIQTOe0EvBnrTfzSniErOHv5ariwADibVxdheUWhTThe454pl4loaiDzxSPDpJqZLW JuYA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id l138si26929539qke.400.2017.08.01.08.02.59; Tue, 01 Aug 2017 08:03:00 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 9598665BE8; Tue, 1 Aug 2017 15:02:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 2A45C60C1E; Tue, 1 Aug 2017 15:01:15 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id DD7C160BD7; Tue, 1 Aug 2017 15:01:08 +0000 (UTC) Received: from forward4h.cmail.yandex.net (forward4h.cmail.yandex.net [87.250.230.101]) by lists.linaro.org (Postfix) with ESMTPS id DDD636099C for ; Tue, 1 Aug 2017 15:00:35 +0000 (UTC) Received: from smtp2j.mail.yandex.net (smtp2j.mail.yandex.net [95.108.130.60]) by forward4h.cmail.yandex.net (Yandex) with ESMTP id 8661C20BD3 for ; Tue, 1 Aug 2017 18:00:34 +0300 (MSK) Received: from smtp2j.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp2j.mail.yandex.net (Yandex) with ESMTP id D85423EC11A0 for ; Tue, 1 Aug 2017 18:00:30 +0300 (MSK) Received: by smtp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id tyzQTD0Qgu-0Tbqs3Ui; Tue, 01 Aug 2017 18:00:29 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) X-Yandex-Suid-Status: 1 0 From: Github ODP bot To: lng-odp@lists.linaro.org Date: Tue, 1 Aug 2017 18:00:05 +0300 Message-Id: <1501599610-28388-6-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> References: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 90 Subject: [lng-odp] [PATCH v3 5/10] linux-generic: packet: unshared_len maintenance X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Bill Fischofer Add code to initialize and maintain unshared_len packet metadata Signed-off-by: Bill Fischofer --- /** Email created from pull request 90 (Bill-Fischofer-Linaro:pktrefs) ** https://github.com/Linaro/odp/pull/90 ** Patch: https://github.com/Linaro/odp/pull/90.patch ** Base sha: bb0ca193f8c28036fdf8b3d6c8aa5d4f11980e6c ** Merge commit sha: dfef320840f96825686124135a6e509db80b7b82 **/ .../linux-generic/include/odp_packet_internal.h | 5 +++ platform/linux-generic/odp_packet.c | 40 ++++++++++++++++------ 2 files changed, 35 insertions(+), 10 deletions(-) diff --git a/platform/linux-generic/include/odp_packet_internal.h b/platform/linux-generic/include/odp_packet_internal.h index 2207ac58..1edb5b7d 100644 --- a/platform/linux-generic/include/odp_packet_internal.h +++ b/platform/linux-generic/include/odp_packet_internal.h @@ -19,6 +19,7 @@ extern "C" { #include #include +#include #include #include #include @@ -250,6 +251,7 @@ static inline void pull_tail(odp_packet_hdr_t *pkt_hdr, uint32_t len) pkt_hdr->tailroom += len; pkt_hdr->frame_len -= len; + pkt_hdr->unshared_len -= len; pkt_hdr->buf_hdr.seg[last].len -= len; } @@ -279,7 +281,10 @@ static inline void packet_ref_count_set(odp_packet_hdr_t *pkt_hdr, uint32_t n) static inline void packet_set_len(odp_packet_hdr_t *pkt_hdr, uint32_t len) { + ODP_ASSERT(packet_ref_count(pkt_hdr) == 1); + pkt_hdr->frame_len = len; + pkt_hdr->unshared_len = len; } /* Forward declarations */ diff --git a/platform/linux-generic/odp_packet.c b/platform/linux-generic/odp_packet.c index 41262cc2..5e1b4f8c 100644 --- a/platform/linux-generic/odp_packet.c +++ b/platform/linux-generic/odp_packet.c @@ -152,6 +152,7 @@ static inline void push_head(odp_packet_hdr_t *pkt_hdr, uint32_t len) { pkt_hdr->headroom -= len; pkt_hdr->frame_len += len; + pkt_hdr->unshared_len += len; pkt_hdr->buf_hdr.seg[0].data -= len; pkt_hdr->buf_hdr.seg[0].len += len; } @@ -160,6 +161,7 @@ static inline void pull_head(odp_packet_hdr_t *pkt_hdr, uint32_t len) { pkt_hdr->headroom += len; pkt_hdr->frame_len -= len; + pkt_hdr->unshared_len -= len; pkt_hdr->buf_hdr.seg[0].data += len; pkt_hdr->buf_hdr.seg[0].len -= len; } @@ -170,6 +172,7 @@ static inline void push_tail(odp_packet_hdr_t *pkt_hdr, uint32_t len) pkt_hdr->tailroom -= len; pkt_hdr->frame_len += len; + pkt_hdr->unshared_len += len; pkt_hdr->buf_hdr.seg[last].len += len; } @@ -197,6 +200,10 @@ static inline void packet_seg_copy_md(odp_packet_hdr_t *dst, dst->buf_hdr.uarea_addr = src->buf_hdr.uarea_addr; dst->buf_hdr.uarea_size = src->buf_hdr.uarea_size; + /* reference related metadata */ + dst->ref_len = src->ref_len; + dst->unshared_len = src->unshared_len; + /* segmentation data is not copied: * buf_hdr.seg[] * buf_hdr.segcount @@ -396,9 +403,10 @@ static inline odp_packet_hdr_t *add_segments(odp_packet_hdr_t *pkt_hdr, new_hdr->buf_hdr.seg[0].len = seg_len; packet_seg_copy_md(new_hdr, pkt_hdr); - new_hdr->frame_len = pkt_hdr->frame_len + len; - new_hdr->headroom = pool->headroom + offset; - new_hdr->tailroom = pkt_hdr->tailroom; + new_hdr->frame_len = pkt_hdr->frame_len + len; + new_hdr->unshared_len = pkt_hdr->unshared_len + len; + new_hdr->headroom = pool->headroom + offset; + new_hdr->tailroom = pkt_hdr->tailroom; pkt_hdr = new_hdr; } else { @@ -411,8 +419,9 @@ static inline odp_packet_hdr_t *add_segments(odp_packet_hdr_t *pkt_hdr, last = packet_last_seg(pkt_hdr); pkt_hdr->buf_hdr.seg[last].len = seg_len; - pkt_hdr->frame_len += len; - pkt_hdr->tailroom = pool->tailroom + offset; + pkt_hdr->frame_len += len; + pkt_hdr->unshared_len += len; + pkt_hdr->tailroom = pool->tailroom + offset; } return pkt_hdr; @@ -450,9 +459,10 @@ static inline odp_packet_hdr_t *free_segments(odp_packet_hdr_t *pkt_hdr, packet_seg_copy_md(new_hdr, pkt_hdr); /* Tailroom not changed */ - new_hdr->tailroom = pkt_hdr->tailroom; - new_hdr->headroom = seg_headroom(new_hdr, 0); - new_hdr->frame_len = pkt_hdr->frame_len - free_len; + new_hdr->tailroom = pkt_hdr->tailroom; + new_hdr->headroom = seg_headroom(new_hdr, 0); + new_hdr->frame_len = pkt_hdr->frame_len - free_len; + new_hdr->unshared_len = pkt_hdr->unshared_len - free_len; pull_head(new_hdr, pull_len); @@ -467,6 +477,7 @@ static inline odp_packet_hdr_t *free_segments(odp_packet_hdr_t *pkt_hdr, * of the metadata. */ pkt_hdr->buf_hdr.segcount = num_remain; pkt_hdr->frame_len -= free_len; + pkt_hdr->unshared_len -= free_len; pkt_hdr->tailroom = seg_tailroom(pkt_hdr, num_remain - 1); pull_tail(pkt_hdr, pull_len); @@ -912,6 +923,7 @@ int odp_packet_extend_head(odp_packet_t *pkt, uint32_t len, pkt_hdr->buf_hdr.segcount = segs; pkt_hdr->frame_len = frame_len; + pkt_hdr->unshared_len = frame_len; pkt_hdr->headroom = offset + pool->headroom; pkt_hdr->tailroom = pool->tailroom; @@ -953,6 +965,8 @@ void *odp_packet_pull_head(odp_packet_t pkt, uint32_t len) if (len > pkt_hdr->frame_len) return NULL; + ODP_ASSERT(len <= pkt_hdr->unshared_len); + pull_head(pkt_hdr, len); return packet_data(pkt_hdr); } @@ -1319,6 +1333,8 @@ int odp_packet_add_data(odp_packet_t *pkt_ptr, uint32_t offset, uint32_t len) if (offset > pktlen) return -1; + ODP_ASSERT(odp_packet_unshared_len(*pkt_ptr) >= offset); + newpkt = odp_packet_alloc(pool->pool_hdl, pktlen + len); if (newpkt == ODP_PACKET_INVALID) @@ -1439,8 +1455,9 @@ int odp_packet_concat(odp_packet_t *dst, odp_packet_t src) add_all_segs(dst_hdr, src_hdr); - dst_hdr->frame_len = dst_len + src_len; - dst_hdr->tailroom = src_hdr->tailroom; + dst_hdr->frame_len = dst_len + src_len; + dst_hdr->unshared_len = dst_len + src_len; + dst_hdr->tailroom = src_hdr->tailroom; /* Data was not moved in memory */ return 0; @@ -1453,6 +1470,7 @@ int odp_packet_split(odp_packet_t *pkt, uint32_t len, odp_packet_t *tail) if (len >= pktlen || tail == NULL) return -1; + ODP_ASSERT(odp_packet_unshared_len(*pkt) >= len); *tail = odp_packet_copy_part(*pkt, len, pktlen - len, odp_packet_pool(*pkt)); @@ -1538,6 +1556,8 @@ int odp_packet_copy_from_mem(odp_packet_t pkt, uint32_t offset, if (offset + len > pkt_hdr->frame_len) return -1; + ODP_ASSERT(odp_packet_unshared_len(pkt) >= offset + len); + while (len > 0) { mapaddr = packet_map(pkt_hdr, offset, &seglen, NULL); cpylen = len > seglen ? seglen : len; From patchwork Tue Aug 1 15:00:06 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 109179 Delivered-To: patch@linaro.org Received: by 10.140.101.6 with SMTP id t6csp1612378qge; Tue, 1 Aug 2017 08:06:48 -0700 (PDT) X-Received: by 10.55.52.73 with SMTP id b70mr25804942qka.74.1501600008769; Tue, 01 Aug 2017 08:06:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501600008; cv=none; d=google.com; s=arc-20160816; b=zS0aBEfM6zih5dSoBB7XKmyeZAogUbCX2PeyE/qTe1aKY+AgNFIvEQFsnX9+sGHZ/U JhjN/fsesdVXtZiSJgTCPm+v/gpL9FDwb9ug4Y9mgOKlRDYKCEd9waBAn/5tERDa1A6K AcURnEdpo2NXWjgmFOEr1DT59wtp8jxi133Xx9Vkb6YqLtN6j2N2NUqLGv8p8TaHftd7 RG9383Q0DnUe+A8IPI+BNKugOaiM7UsZOFKFQ+weSOAM97bOicjt/qGxeKeYk2nJedzU CRKJQVAJD8JAWGbNJnzmzZdNXPKm0+W37I/lrCnnY8HVtGKhGZm9pyWWxLUJOBIUo3cG cpoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=SBr74wvWhWIyiiflUNCBmgiApbO769BUpKJH6VBgbKg=; b=TxJ7GaiJnJADhmuT1N3HoKijat4ZZxhYd4vcY8bJgkHYLV8fCNTAF7Y8YD+fXqOfxb ENf2rgMO9FD+unuNZ4XWZVVC9U1UZ3qBLzngzoKwV90dIrS5zQXfdPy59D80EekVhG5I fLFCOg8aMkZYdgxd2O6MBIz7AG3mAHWwdbjv4PpJxEGbX/mJzI2QS4aBDTX+iqwiV6Rp IIVV5S+CeR6TKS+UKJzTvtn+Z7ONV5gt/FqLax/Y2tRMXU0zpPXcqtC6Ps460iWNaDqr 9apzVnT5EIGluASwBod4taB4oB85eDVtNVOTxkJrK19jLkMaSgZrmC6F8AJSeRkcnma2 4v5w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id b72si26307562qkg.336.2017.08.01.08.06.48; Tue, 01 Aug 2017 08:06:48 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 6345F65BCC; Tue, 1 Aug 2017 15:06:48 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id CA5B965BD2; Tue, 1 Aug 2017 15:01:40 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 09ACE644E6; Tue, 1 Aug 2017 15:01:17 +0000 (UTC) Received: from forward3m.cmail.yandex.net (forward3m.cmail.yandex.net [5.255.216.21]) by lists.linaro.org (Postfix) with ESMTPS id 8950D644CB for ; Tue, 1 Aug 2017 15:00:36 +0000 (UTC) Received: from smtp2j.mail.yandex.net (smtp2j.mail.yandex.net [IPv6:2a02:6b8:0:801::ac]) by forward3m.cmail.yandex.net (Yandex) with ESMTP id 46A5421468 for ; Tue, 1 Aug 2017 18:00:35 +0300 (MSK) Received: from smtp2j.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp2j.mail.yandex.net (Yandex) with ESMTP id 516A93EC1198 for ; Tue, 1 Aug 2017 18:00:31 +0300 (MSK) Received: by smtp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id tyzQTD0Qgu-0Vbi8r5p; Tue, 01 Aug 2017 18:00:31 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) X-Yandex-Suid-Status: 1 0 From: Github ODP bot To: lng-odp@lists.linaro.org Date: Tue, 1 Aug 2017 18:00:06 +0300 Message-Id: <1501599610-28388-7-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> References: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 90 Subject: [lng-odp] [PATCH v3 6/10] linux-generic: packet: segment manipulation with references X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Bill Fischofer Add support for segment operation with references Signed-off-by: Bill Fischofer --- /** Email created from pull request 90 (Bill-Fischofer-Linaro:pktrefs) ** https://github.com/Linaro/odp/pull/90 ** Patch: https://github.com/Linaro/odp/pull/90.patch ** Base sha: bb0ca193f8c28036fdf8b3d6c8aa5d4f11980e6c ** Merge commit sha: dfef320840f96825686124135a6e509db80b7b82 **/ .../include/odp/api/plat/packet_inlines.h | 40 ++--- .../include/odp/api/plat/packet_inlines_api.h | 16 -- .../linux-generic/include/odp_packet_internal.h | 55 ++++++- platform/linux-generic/odp_packet.c | 170 +++++++++++++++++---- 4 files changed, 212 insertions(+), 69 deletions(-) diff --git a/platform/linux-generic/include/odp/api/plat/packet_inlines.h b/platform/linux-generic/include/odp/api/plat/packet_inlines.h index d0cf1390..06b049fc 100644 --- a/platform/linux-generic/include/odp/api/plat/packet_inlines.h +++ b/platform/linux-generic/include/odp/api/plat/packet_inlines.h @@ -50,7 +50,19 @@ static inline uint32_t _odp_packet_seg_len(odp_packet_t pkt) /** @internal Inline function @param pkt @return */ static inline uint32_t _odp_packet_len(odp_packet_t pkt) { - return _odp_pkt_get(pkt, uint32_t, frame_len); + uint32_t pkt_len = _odp_pkt_get(pkt, uint32_t, frame_len); + void *ref_nxt = _odp_pkt_get(pkt, void *, ref_hdr); + void *ref_pkt = (void *)pkt; + + while (ref_nxt) { + pkt_len += _odp_pkt_get(ref_pkt, uint32_t, ref_len) - + _odp_pkt_get(ref_pkt, uint32_t, ref_offset); + + ref_pkt = ref_nxt; + ref_nxt = _odp_pkt_get(ref_nxt, void *, ref_hdr); + } + + return pkt_len; } /** @internal Inline function @param pkt @return */ @@ -78,12 +90,6 @@ static inline odp_pktio_t _odp_packet_input(odp_packet_t pkt) } /** @internal Inline function @param pkt @return */ -static inline int _odp_packet_num_segs(odp_packet_t pkt) -{ - return _odp_pkt_get(pkt, uint8_t, segcount); -} - -/** @internal Inline function @param pkt @return */ static inline void *_odp_packet_user_ptr(odp_packet_t pkt) { return _odp_pkt_get(pkt, void *, user_ptr); @@ -122,7 +128,8 @@ static inline void *_odp_packet_head(odp_packet_t pkt) /** @internal Inline function @param pkt @return */ static inline int _odp_packet_is_segmented(odp_packet_t pkt) { - return _odp_pkt_get(pkt, uint8_t, segcount) > 1; + return _odp_pkt_get(pkt, uint8_t, segcount) > 1 || + _odp_pkt_get(pkt, void *, ref_hdr) != NULL; } /** @internal Inline function @param pkt @return */ @@ -133,23 +140,6 @@ static inline odp_packet_seg_t _odp_packet_first_seg(odp_packet_t pkt) return _odp_packet_seg_from_ndx(0); } -/** @internal Inline function @param pkt @return */ -static inline odp_packet_seg_t _odp_packet_last_seg(odp_packet_t pkt) -{ - return _odp_packet_seg_from_ndx(_odp_packet_num_segs(pkt) - 1); -} - -/** @internal Inline function @param pkt @param seg @return */ -static inline odp_packet_seg_t _odp_packet_next_seg(odp_packet_t pkt, - odp_packet_seg_t seg) -{ - if (odp_unlikely(_odp_packet_seg_to_ndx(seg) >= - _odp_packet_seg_to_ndx(_odp_packet_last_seg(pkt)))) - return ODP_PACKET_SEG_INVALID; - - return seg + 1; -} - /** @internal Inline function @param pkt @param offset @param len */ static inline void _odp_packet_prefetch(odp_packet_t pkt, uint32_t offset, uint32_t len) diff --git a/platform/linux-generic/include/odp/api/plat/packet_inlines_api.h b/platform/linux-generic/include/odp/api/plat/packet_inlines_api.h index 233bc876..f818f820 100644 --- a/platform/linux-generic/include/odp/api/plat/packet_inlines_api.h +++ b/platform/linux-generic/include/odp/api/plat/packet_inlines_api.h @@ -48,11 +48,6 @@ _ODP_INLINE odp_pktio_t odp_packet_input(odp_packet_t pkt) return _odp_packet_input(pkt); } -_ODP_INLINE int odp_packet_num_segs(odp_packet_t pkt) -{ - return _odp_packet_num_segs(pkt); -} - _ODP_INLINE void *odp_packet_user_ptr(odp_packet_t pkt) { return _odp_packet_user_ptr(pkt); @@ -93,17 +88,6 @@ _ODP_INLINE odp_packet_seg_t odp_packet_first_seg(odp_packet_t pkt) return _odp_packet_first_seg(pkt); } -_ODP_INLINE odp_packet_seg_t odp_packet_last_seg(odp_packet_t pkt) -{ - return _odp_packet_last_seg(pkt); -} - -_ODP_INLINE odp_packet_seg_t odp_packet_next_seg(odp_packet_t pkt, - odp_packet_seg_t seg) -{ - return _odp_packet_next_seg(pkt, seg); -} - _ODP_INLINE void odp_packet_prefetch(odp_packet_t pkt, uint32_t offset, uint32_t len) { diff --git a/platform/linux-generic/include/odp_packet_internal.h b/platform/linux-generic/include/odp_packet_internal.h index 1edb5b7d..d923ee23 100644 --- a/platform/linux-generic/include/odp_packet_internal.h +++ b/platform/linux-generic/include/odp_packet_internal.h @@ -171,6 +171,50 @@ static inline odp_packet_hdr_t *odp_packet_hdr(odp_packet_t pkt) return (odp_packet_hdr_t *)(uintptr_t)pkt; } +static inline odp_packet_hdr_t *packet_last_hdr(odp_packet_t pkt, + uint32_t *offset) +{ + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + odp_packet_hdr_t *prev_hdr = pkt_hdr; + uint32_t ref_offset = 0; + + while (pkt_hdr->ref_hdr) { + ref_offset = pkt_hdr->ref_offset; + prev_hdr = pkt_hdr; + pkt_hdr = pkt_hdr->ref_hdr; + } + + if (offset) { + if (prev_hdr != pkt_hdr) + ref_offset += pkt_hdr->frame_len - prev_hdr->ref_len; + *offset = ref_offset; + } + + return pkt_hdr; +} + +static inline odp_packet_hdr_t *packet_prev_hdr(odp_packet_hdr_t *pkt_hdr, + odp_packet_hdr_t *cur_hdr, + uint32_t *offset) +{ + uint32_t ref_offset = 0; + odp_packet_hdr_t *prev_hdr = pkt_hdr; + + while (pkt_hdr->ref_hdr != cur_hdr) { + ref_offset = pkt_hdr->ref_offset; + prev_hdr = pkt_hdr; + pkt_hdr = pkt_hdr->ref_hdr; + } + + if (offset) { + if (prev_hdr != pkt_hdr) + ref_offset += pkt_hdr->frame_len - prev_hdr->ref_len; + *offset = ref_offset; + } + + return pkt_hdr; +} + static inline odp_packet_t packet_handle(odp_packet_hdr_t *pkt_hdr) { return (odp_packet_t)pkt_hdr; @@ -257,7 +301,16 @@ static inline void pull_tail(odp_packet_hdr_t *pkt_hdr, uint32_t len) static inline uint32_t packet_len(odp_packet_hdr_t *pkt_hdr) { - return pkt_hdr->frame_len; + uint32_t pkt_len = pkt_hdr->frame_len; + odp_packet_hdr_t *ref_hdr = pkt_hdr->ref_hdr; + + while (ref_hdr) { + pkt_len += (pkt_hdr->ref_len - pkt_hdr->ref_offset); + pkt_hdr = ref_hdr; + ref_hdr = ref_hdr->ref_hdr; + } + + return pkt_len; } static inline uint32_t packet_ref_count(odp_packet_hdr_t *pkt_hdr) diff --git a/platform/linux-generic/odp_packet.c b/platform/linux-generic/odp_packet.c index 5e1b4f8c..bfe9f795 100644 --- a/platform/linux-generic/odp_packet.c +++ b/platform/linux-generic/odp_packet.c @@ -92,12 +92,13 @@ static inline uint32_t packet_seg_len(odp_packet_hdr_t *pkt_hdr, return pkt_hdr->buf_hdr.seg[seg_idx].len; } -static inline void *packet_seg_data(odp_packet_hdr_t *pkt_hdr, uint32_t seg_idx) +static inline uint8_t *packet_seg_data(odp_packet_hdr_t *pkt_hdr, + uint32_t seg_idx) { return pkt_hdr->buf_hdr.seg[seg_idx].data; } -static inline int packet_last_seg(odp_packet_hdr_t *pkt_hdr) +static inline uint32_t packet_last_seg(odp_packet_hdr_t *pkt_hdr) { if (CONFIG_PACKET_MAX_SEGS == 1) return 0; @@ -218,7 +219,15 @@ static inline void *packet_map(odp_packet_hdr_t *pkt_hdr, int seg = 0; int seg_count = pkt_hdr->buf_hdr.segcount; - if (odp_unlikely(offset >= pkt_hdr->frame_len)) + /* Special processing for references */ + while (offset >= pkt_hdr->frame_len && pkt_hdr->ref_hdr) { + offset -= (pkt_hdr->frame_len - pkt_hdr->ref_offset); + offset += (pkt_hdr->ref_hdr->frame_len - pkt_hdr->ref_len); + pkt_hdr = pkt_hdr->ref_hdr; + seg_count = pkt_hdr->buf_hdr.segcount; + } + + if (odp_unlikely(offset > pkt_hdr->frame_len)) return NULL; if (odp_likely(CONFIG_PACKET_MAX_SEGS == 1 || seg_count == 1)) { @@ -670,13 +679,47 @@ odp_event_t odp_packet_to_event(odp_packet_t pkt) uint32_t odp_packet_buf_len(odp_packet_t pkt) { odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt); + uint32_t buf_len = 0; - return pkt_hdr->buf_hdr.size * pkt_hdr->buf_hdr.segcount; + do { + buf_len += pkt_hdr->buf_hdr.size * pkt_hdr->buf_hdr.segcount; + pkt_hdr = pkt_hdr->ref_hdr; + } while (pkt_hdr); + + return buf_len; } -void *odp_packet_tail(odp_packet_t pkt) +uint32_t odp_packet_unshared_len(odp_packet_t pkt) { odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt); + uint32_t pkt_len = 0, offset = 0; + + if (packet_ref_count(pkt_hdr) == 1) + pkt_hdr->unshared_len = pkt_hdr->frame_len; + + do { + if (packet_ref_count(pkt_hdr) > 1) { + if (offset == 0) + pkt_len += pkt_hdr->unshared_len; + break; + } + + pkt_len += pkt_hdr->frame_len - offset; + offset = pkt_hdr->ref_offset; + + if (pkt_hdr->ref_hdr) + offset += (pkt_hdr->ref_hdr->frame_len - + pkt_hdr->ref_len); + + pkt_hdr = pkt_hdr->ref_hdr; + } while (pkt_hdr); + + return pkt_len; +} + +void *odp_packet_tail(odp_packet_t pkt) +{ + odp_packet_hdr_t *pkt_hdr = packet_last_hdr(pkt, NULL); return packet_tail(pkt_hdr); } @@ -1286,6 +1329,46 @@ void odp_packet_ts_set(odp_packet_t pkt, odp_time_t timestamp) pkt_hdr->p.input_flags.timestamp = 1; } +int odp_packet_num_segs(odp_packet_t pkt) +{ + odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt); + uint32_t segcount = 0, i; + uint32_t seg_offset = 0, offset; + + do { + segcount += pkt_hdr->buf_hdr.segcount - seg_offset; + offset = pkt_hdr->ref_offset; + pkt_hdr = pkt_hdr->ref_hdr; + if (pkt_hdr) { + for (i = 0, seg_offset = 0; + i < pkt_hdr->buf_hdr.segcount; + i++, seg_offset++) { + if (offset < pkt_hdr->buf_hdr.seg[i].len) + break; + offset -= pkt_hdr->buf_hdr.seg[i].len; + } + } + } while (pkt_hdr); + + return segcount; +} + +odp_packet_seg_t odp_packet_last_seg(odp_packet_t pkt) +{ + return _odp_packet_seg_from_ndx(odp_packet_num_segs(pkt) - 1); +} + +odp_packet_seg_t odp_packet_next_seg(odp_packet_t pkt, odp_packet_seg_t seg) +{ + odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt); + + if (odp_unlikely(_odp_packet_seg_to_ndx(seg) >= + packet_last_seg(pkt_hdr))) + return ODP_PACKET_SEG_INVALID; + + return seg + 1; +} + /* * * Segment level @@ -1296,23 +1379,53 @@ void odp_packet_ts_set(odp_packet_t pkt, odp_time_t timestamp) void *odp_packet_seg_data(odp_packet_t pkt, odp_packet_seg_t seg) { odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt); + uint32_t seg_offset = 0, offset = 0, i; + uint32_t seg_ndx = _odp_packet_seg_to_ndx(seg); + + while (seg_ndx >= pkt_hdr->buf_hdr.segcount - seg_offset && + pkt_hdr->ref_hdr) { + seg_ndx -= (pkt_hdr->buf_hdr.segcount - seg_offset); + offset = pkt_hdr->ref_offset; + pkt_hdr = pkt_hdr->ref_hdr; + for (i = 0, seg_offset = 0; + i < pkt_hdr->buf_hdr.segcount; + i++, seg_offset++) { + if (offset < pkt_hdr->buf_hdr.seg[i].len) + break; + offset -= pkt_hdr->buf_hdr.seg[i].len; + } + } - if (odp_unlikely(_odp_packet_seg_to_ndx(seg) >= - pkt_hdr->buf_hdr.segcount)) + if (odp_unlikely(seg_ndx + seg_offset >= pkt_hdr->buf_hdr.segcount)) return NULL; - return packet_seg_data(pkt_hdr, _odp_packet_seg_to_ndx(seg)); + return packet_seg_data(pkt_hdr, seg_ndx + seg_offset) + offset; } uint32_t odp_packet_seg_data_len(odp_packet_t pkt, odp_packet_seg_t seg) { odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt); + uint32_t seg_offset = 0, offset = 0, i; + uint32_t seg_ndx = _odp_packet_seg_to_ndx(seg); + + while (seg_ndx >= pkt_hdr->buf_hdr.segcount - seg_offset && + pkt_hdr->ref_hdr) { + seg_ndx -= (pkt_hdr->buf_hdr.segcount - seg_offset); + offset = pkt_hdr->ref_offset; + pkt_hdr = pkt_hdr->ref_hdr; + for (i = 0, seg_offset = 0; + i < pkt_hdr->buf_hdr.segcount; + i++, seg_offset++) { + if (offset < pkt_hdr->buf_hdr.seg[i].len) + break; + offset -= pkt_hdr->buf_hdr.seg[i].len; + } + } - if (odp_unlikely(_odp_packet_seg_to_ndx(seg) >= - pkt_hdr->buf_hdr.segcount)) + if (odp_unlikely(seg_ndx + seg_offset >= pkt_hdr->buf_hdr.segcount)) return 0; - return packet_seg_len(pkt_hdr, _odp_packet_seg_to_ndx(seg)); + return packet_seg_len(pkt_hdr, seg_ndx + seg_offset) - offset; } /* @@ -1358,13 +1471,15 @@ int odp_packet_rem_data(odp_packet_t *pkt_ptr, uint32_t offset, uint32_t len) { odp_packet_t pkt = *pkt_ptr; odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt); - uint32_t pktlen = pkt_hdr->frame_len; + uint32_t pktlen = packet_len(pkt_hdr); pool_t *pool = pkt_hdr->buf_hdr.pool_ptr; odp_packet_t newpkt; - if (offset > pktlen || offset + len > pktlen) + if (offset + len > pktlen) return -1; + ODP_ASSERT(odp_packet_unshared_len(*pkt_ptr) >= offset + len); + newpkt = odp_packet_alloc(pool->pool_hdl, pktlen - len); if (newpkt == ODP_PACKET_INVALID) @@ -1398,6 +1513,8 @@ int odp_packet_align(odp_packet_t *pkt, uint32_t offset, uint32_t len, if (align > ODP_CACHE_LINE_SIZE) return -1; + ODP_ASSERT(odp_packet_has_ref(*pkt) == 0); + if (seglen >= len) { misalign = align <= 1 ? 0 : ROUNDUP_ALIGN(uaddr, align) - uaddr; @@ -1437,10 +1554,13 @@ int odp_packet_concat(odp_packet_t *dst, odp_packet_t src) uint32_t dst_len = dst_hdr->frame_len; uint32_t src_len = src_hdr->frame_len; + ODP_ASSERT(packet_ref_count(dst_hdr) == 1); + /* Do a copy if resulting packet would be out of segments or packets - * are from different pools. */ + * are from different pools or src is a reference. */ if (odp_unlikely((dst_segs + src_segs) > CONFIG_PACKET_MAX_SEGS) || - odp_unlikely(dst_pool != src_pool)) { + odp_unlikely(dst_pool != src_pool) || + odp_unlikely(packet_ref_count(src_hdr) > 1)) { if (odp_packet_extend_tail(dst, src_len, NULL, NULL) >= 0) { (void)odp_packet_copy_from_pkt(*dst, dst_len, src, 0, src_len); @@ -1489,8 +1609,7 @@ int odp_packet_split(odp_packet_t *pkt, uint32_t len, odp_packet_t *tail) odp_packet_t odp_packet_copy(odp_packet_t pkt, odp_pool_t pool) { - odp_packet_hdr_t *srchdr = packet_hdr(pkt); - uint32_t pktlen = srchdr->frame_len; + uint32_t pktlen = odp_packet_len(pkt); odp_packet_t newpkt = odp_packet_alloc(pool, pktlen); if (newpkt != ODP_PACKET_INVALID) { @@ -1529,7 +1648,7 @@ int odp_packet_copy_to_mem(odp_packet_t pkt, uint32_t offset, uint8_t *dstaddr = (uint8_t *)dst; odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt); - if (offset + len > pkt_hdr->frame_len) + if (offset + len > packet_len(pkt_hdr)) return -1; while (len > 0) { @@ -1553,7 +1672,7 @@ int odp_packet_copy_from_mem(odp_packet_t pkt, uint32_t offset, const uint8_t *srcaddr = (const uint8_t *)src; odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt); - if (offset + len > pkt_hdr->frame_len) + if (offset + len > packet_len(pkt_hdr)) return -1; ODP_ASSERT(odp_packet_unshared_len(pkt) >= offset + len); @@ -1583,10 +1702,12 @@ int odp_packet_copy_from_pkt(odp_packet_t dst, uint32_t dst_offset, uint32_t src_seglen = 0; /* GCC */ int overlap; - if (dst_offset + len > dst_hdr->frame_len || - src_offset + len > src_hdr->frame_len) + if (dst_offset + len > packet_len(dst_hdr) || + src_offset + len > packet_len(src_hdr)) return -1; + ODP_ASSERT(odp_packet_unshared_len(dst) >= dst_offset + len); + overlap = (dst_hdr == src_hdr && ((dst_offset <= src_offset && dst_offset + len >= src_offset) || @@ -1719,7 +1840,7 @@ void odp_packet_print(odp_packet_t pkt) len += snprintf(&str[len], n - len, " l4_offset %" PRIu32 "\n", hdr->p.l4_offset); len += snprintf(&str[len], n - len, - " frame_len %" PRIu32 "\n", hdr->frame_len); + " frame_len %" PRIu32 "\n", packet_len(hdr)); len += snprintf(&str[len], n - len, " input %" PRIu64 "\n", odp_pktio_to_u64(hdr->input)); @@ -2192,11 +2313,6 @@ int odp_packet_has_ref(odp_packet_t pkt) return 0; } -uint32_t odp_packet_unshared_len(odp_packet_t pkt) -{ - return odp_packet_len(pkt); -} - /* Include non-inlined versions of API functions */ #if ODP_ABI_COMPAT == 1 #include From patchwork Tue Aug 1 15:00:07 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 109178 Delivered-To: patch@linaro.org Received: by 10.140.101.6 with SMTP id t6csp1611141qge; Tue, 1 Aug 2017 08:05:54 -0700 (PDT) X-Received: by 10.200.38.74 with SMTP id v10mr27301386qtv.95.1501599954376; Tue, 01 Aug 2017 08:05:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501599954; cv=none; d=google.com; s=arc-20160816; b=NvWYsU+WlMQErgON4OniaVNEdStr3bD+KdLaYbBEBTtLvDT+5Fw0H5irlZzw9yp6xV Tqi3iL0AauAvw7Xsq0wmO4cJPDwEFRekZ7qsaalurqP8Qg6jrmihfd+F2Ge2ndDHie3G Ym65KE5AgjnqWf8ueqZ+uKPBJOOe1fSnHjl5NJESLfkolR/Pni3efN9UxvOVXxbA+R+m x5cetn1iUvBR1cfdKBsIAa/VTlqHbsaV4v1KN6j8qEI6gln+nrPssxv4dPzED3xwIigN y5krKzDIc3WawTOfa5Sk8/rkRcPc209zf0LcuTRqCju777XE6Jbqxxqzr9+cC9oxakyt Kp9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=jjODF+bF+oRxtDBjtDUWE/Dv5Kz2Q2CxqU84iLhin4o=; b=KIUZqzWln++5hwiiTmzsjAeDg2WH37X6wvUOJIdC4oGM4FhUGgnD7qMRHz2IOu0/Js mwLWI0hlgnb34PyMU1qJESWhS6rU4SIhRFwdwG/0zpkRhbiA93mle5n7PGe559GcQdf6 +GXLznEIlWGBODKxDEfayh0cFOwYVY5Gzlev+k0F/sBftKg0A0UiBWteNfQJC+S90U/7 eKxyUzf3XnU4yNzW7HQpfs92tw71gXivY/IkiCp2y7hVqs7mwC9i9T7mkr1Dm2rA7iyt WxMaNPIdJsS7HpFUAyBTY4LR/HODQDe5ucPJT84Era8O4oXLm6ejHcT5Loa0RBKUpgX9 fmFQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id a12si21285807qtd.334.2017.08.01.08.05.54; Tue, 01 Aug 2017 08:05:54 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id B95CC6084F; Tue, 1 Aug 2017 15:05:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 5547365BCF; Tue, 1 Aug 2017 15:01:34 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 8122F644E6; Tue, 1 Aug 2017 15:01:15 +0000 (UTC) Received: from forward5o.cmail.yandex.net (forward5o.cmail.yandex.net [37.9.109.249]) by lists.linaro.org (Postfix) with ESMTPS id 3D4DF60B4E for ; Tue, 1 Aug 2017 15:00:37 +0000 (UTC) Received: from smtp2j.mail.yandex.net (smtp2j.mail.yandex.net [IPv6:2a02:6b8:0:801::ac]) by forward5o.cmail.yandex.net (Yandex) with ESMTP id 5FA03206C5 for ; Tue, 1 Aug 2017 18:00:35 +0300 (MSK) Received: from smtp2j.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp2j.mail.yandex.net (Yandex) with ESMTP id B4EA33EC10D0 for ; Tue, 1 Aug 2017 18:00:33 +0300 (MSK) Received: by smtp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id tyzQTD0Qgu-0WbWpWQJ; Tue, 01 Aug 2017 18:00:32 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) X-Yandex-Suid-Status: 1 0 From: Github ODP bot To: lng-odp@lists.linaro.org Date: Tue, 1 Aug 2017 18:00:07 +0300 Message-Id: <1501599610-28388-8-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> References: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 90 Subject: [lng-odp] [PATCH v3 7/10] linux-generic: packet: restructure free routines for references X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Bill Fischofer Restructure the various packet/segment free routines to support references. Signed-off-by: Bill Fischofer --- /** Email created from pull request 90 (Bill-Fischofer-Linaro:pktrefs) ** https://github.com/Linaro/odp/pull/90 ** Patch: https://github.com/Linaro/odp/pull/90.patch ** Base sha: bb0ca193f8c28036fdf8b3d6c8aa5d4f11980e6c ** Merge commit sha: dfef320840f96825686124135a6e509db80b7b82 **/ platform/linux-generic/odp_packet.c | 123 +++++++++++++++++++++++++----------- 1 file changed, 87 insertions(+), 36 deletions(-) diff --git a/platform/linux-generic/odp_packet.c b/platform/linux-generic/odp_packet.c index bfe9f795..4dd84e26 100644 --- a/platform/linux-generic/odp_packet.c +++ b/platform/linux-generic/odp_packet.c @@ -438,13 +438,20 @@ static inline odp_packet_hdr_t *add_segments(odp_packet_hdr_t *pkt_hdr, static inline void free_bufs(odp_packet_hdr_t *pkt_hdr, int first, int num) { - int i; + int i, nfree; odp_buffer_hdr_t *buf_hdr[num]; - for (i = 0; i < num; i++) - buf_hdr[i] = pkt_hdr->buf_hdr.seg[first + i].hdr; + for (i = 0, nfree = 0; i < num; i++) { + odp_packet_hdr_t *hdr = pkt_hdr->buf_hdr.seg[first + i].hdr; + + if (packet_ref_count(hdr) == 1 || packet_ref_dec(hdr) == 1) { + ODP_ASSERT((packet_ref_count_set(hdr, 0), 1)); + buf_hdr[nfree++] = &hdr->buf_hdr; + } + } - buffer_free_multi(buf_hdr, num); + if (nfree > 0) + buffer_free_multi(buf_hdr, nfree); } static inline odp_packet_hdr_t *free_segments(odp_packet_hdr_t *pkt_hdr, @@ -455,11 +462,19 @@ static inline odp_packet_hdr_t *free_segments(odp_packet_hdr_t *pkt_hdr, if (head) { odp_packet_hdr_t *new_hdr; - int i; + int i, nfree; odp_buffer_hdr_t *buf_hdr[num]; - for (i = 0; i < num; i++) - buf_hdr[i] = pkt_hdr->buf_hdr.seg[i].hdr; + for (i = 0, nfree = 0; i < num; i++) { + new_hdr = pkt_hdr->buf_hdr.seg[i].hdr; + + if (packet_ref_count(new_hdr) == 1 || + packet_ref_dec(new_hdr) == 1) { + ODP_ASSERT((packet_ref_count_set(new_hdr, 0), + 1)); + buf_hdr[nfree++] = &new_hdr->buf_hdr; + } + } /* First remaining segment is the new packet descriptor */ new_hdr = pkt_hdr->buf_hdr.seg[num].hdr; @@ -477,7 +492,8 @@ static inline odp_packet_hdr_t *free_segments(odp_packet_hdr_t *pkt_hdr, pkt_hdr = new_hdr; - buffer_free_multi(buf_hdr, num); + if (nfree > 0) + buffer_free_multi(buf_hdr, nfree); } else { /* Free last 'num' bufs */ free_bufs(pkt_hdr, num_remain, num); @@ -593,48 +609,80 @@ int odp_packet_alloc_multi(odp_pool_t pool_hdl, uint32_t len, return num; } -void odp_packet_free(odp_packet_t pkt) +static inline void packet_free(odp_packet_hdr_t *pkt_hdr) { - odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt); - odp_buffer_t hdl = buffer_handle(pkt_hdr); + odp_packet_hdr_t *ref_hdr; + odp_buffer_hdr_t *buf_hdr; + uint32_t ref_count; + int num_seg; + + do { + buf_hdr = &pkt_hdr->buf_hdr; + ref_count = packet_ref_count(pkt_hdr); + num_seg = pkt_hdr->buf_hdr.segcount; + ref_hdr = pkt_hdr->ref_hdr; + ODP_ASSERT(ref_count >= 1); + + if (odp_likely((CONFIG_PACKET_MAX_SEGS == 1 || num_seg == 1) && + ref_count == 1)) { + ODP_ASSERT((packet_ref_count_set(pkt_hdr, 0), 1)); + buffer_free_multi(&buf_hdr, 1); + } else { + free_bufs(pkt_hdr, 0, num_seg); + } - int num_seg = pkt_hdr->buf_hdr.segcount; + pkt_hdr = ref_hdr; + } while (pkt_hdr); +} - if (odp_likely(CONFIG_PACKET_MAX_SEGS == 1 || num_seg == 1)) - buffer_free_multi((odp_buffer_hdr_t **)&hdl, 1); - else - free_bufs(pkt_hdr, 0, num_seg); +void odp_packet_free(odp_packet_t pkt) +{ + packet_free(packet_hdr(pkt)); } void odp_packet_free_multi(const odp_packet_t pkt[], int num) { - if (CONFIG_PACKET_MAX_SEGS == 1) { - buffer_free_multi((odp_buffer_hdr_t **)(uintptr_t)pkt, num); - } else { - odp_buffer_hdr_t *buf_hdr[num * CONFIG_PACKET_MAX_SEGS]; - int i; - int j; - int bufs = 0; + odp_packet_hdr_t *pkt_hdr, *ref_hdr, *hdr; + int nbufs = num * CONFIG_PACKET_MAX_SEGS * 2; + odp_buffer_hdr_t *buf_hdr[nbufs]; + int num_seg; + int i, j; + uint32_t ref_count; + int nfree = 0; - for (i = 0; i < num; i++) { - odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt[i]); - int num_seg = pkt_hdr->buf_hdr.segcount; - odp_buffer_hdr_t *hdr = &pkt_hdr->buf_hdr; + for (i = 0; i < num; i++) { + pkt_hdr = packet_hdr(pkt[i]); - buf_hdr[bufs] = hdr; - bufs++; + do { + num_seg = pkt_hdr->buf_hdr.segcount; + ref_hdr = pkt_hdr->ref_hdr; - if (odp_likely(num_seg == 1)) - continue; + /* Make sure we have enough space for this pkt's segs */ + if (nfree + num_seg > nbufs) { + buffer_free_multi(buf_hdr, nfree); + nfree = 0; + } - for (j = 1; j < num_seg; j++) { - buf_hdr[bufs] = hdr->seg[j].hdr; - bufs++; + for (j = 0; j < num_seg; j++) { + hdr = pkt_hdr->buf_hdr.seg[j].hdr; + ref_count = packet_ref_count(hdr); + ODP_ASSERT(ref_count >= 1); + + if (ref_count == 1 || + packet_ref_dec(hdr) == 1) { + ODP_ASSERT + ((packet_ref_count_set(hdr, 0), + 1)); + buf_hdr[nfree++] = &hdr->buf_hdr; + } } - } - buffer_free_multi(buf_hdr, bufs); + pkt_hdr = ref_hdr; + } while (pkt_hdr); } + + if (nfree > 0) + buffer_free_multi(buf_hdr, nfree); } int odp_packet_reset(odp_packet_t pkt, uint32_t len) @@ -646,6 +694,9 @@ int odp_packet_reset(odp_packet_t pkt, uint32_t len) if (odp_unlikely(len > (pool->max_seg_len * num))) return -1; + if (pkt_hdr->ref_hdr) + packet_free(pkt_hdr->ref_hdr); + reset_seg(pkt_hdr, 0, num); packet_init(pkt_hdr, len); From patchwork Tue Aug 1 15:00:08 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 109180 Delivered-To: patch@linaro.org Received: by 10.140.101.6 with SMTP id t6csp1614641qge; Tue, 1 Aug 2017 08:08:28 -0700 (PDT) X-Received: by 10.200.55.51 with SMTP id o48mr27381643qtb.120.1501600108375; Tue, 01 Aug 2017 08:08:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501600108; cv=none; d=google.com; s=arc-20160816; b=vwCt1bnsqxAzJXkEr+wOoNT0S5/N52VylAESZSAIeKVDFVjQwnaKg19BUxVblsouvG 9uOGGfDr6Pqa5UEK6796yHhA3DZ+EYwovKYFDCao8HVu44MshtPZ9KCzErBqLmtaAu4P sbuov+4gcvSNYoUKcobecrGybKCQbIENNHz8uAfuAtD212ZjgjaAZuAa9qJNv9Db3uBl Eb7C2bFyhE4YNXF1oWdRA+Rl9tdrEhH/2izoX0D3RJ2QHSsoAbcBUuY/H0m/5nG9UjLr /l+hCzCynTpFMFqlnRWYn+R9ckVucmJ0+k60XqM/nxrw6HrMn57VJHMby9UE3e/60HBu AZ5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=2zVYuMukgxm8u2HHeq5suYbQdhhYAWskWigby4LCtOM=; b=SpvHym95RSZ7JP/hoXwWIMbssgX/O+ux7KJiqNOR1d1zWZgEIiWKFWqVBQkFcqIuzD Fipv9KTxF9cFepZiS+3fuWidHMeHA7dsp9DKcJbiPT4LB+zEoZiYap2u+vXgDL8JqHBw xrR2plIMfxBt+UIGYdkbrI6ixVYDfEIkTMVjAy/3f4FwHgBbsk3aoXmRtY7EhJs/Y+gR AXk1pR858MQ2bCngr/dT0w6xuGGmUB5nHWDW+lLVjIoqP97ZSPSuZFE59LWBMrTM3hml Y4VQR6pxeVSB7DlHNUrY0cp6FxQrcjVyk1V4BXJAfzBbUvyyjOwI2/9i/5jdT68N+G5p wMNA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id l35si25500264qte.145.2017.08.01.08.08.28; Tue, 01 Aug 2017 08:08:28 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id D9DEC65BBF; Tue, 1 Aug 2017 15:08:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id B8F4965897; Tue, 1 Aug 2017 15:02:54 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id B393265BCE; Tue, 1 Aug 2017 15:02:50 +0000 (UTC) Received: from forward3m.cmail.yandex.net (forward3m.cmail.yandex.net [5.255.216.21]) by lists.linaro.org (Postfix) with ESMTPS id 83E30644E0 for ; Tue, 1 Aug 2017 15:00:39 +0000 (UTC) Received: from smtp2j.mail.yandex.net (smtp2j.mail.yandex.net [95.108.130.60]) by forward3m.cmail.yandex.net (Yandex) with ESMTP id 7EE5E21244 for ; Tue, 1 Aug 2017 18:00:38 +0300 (MSK) Received: from smtp2j.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp2j.mail.yandex.net (Yandex) with ESMTP id 475A53EC116B for ; Tue, 1 Aug 2017 18:00:34 +0300 (MSK) Received: by smtp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id tyzQTD0Qgu-0YbOut9V; Tue, 01 Aug 2017 18:00:34 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) X-Yandex-Suid-Status: 1 0 From: Github ODP bot To: lng-odp@lists.linaro.org Date: Tue, 1 Aug 2017 18:00:08 +0300 Message-Id: <1501599610-28388-9-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> References: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 90 Subject: [lng-odp] [PATCH v3 8/10] linux-generic: packet: add reference support for head/tail manipulation X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Bill Fischofer Signed-off-by: Bill Fischofer --- /** Email created from pull request 90 (Bill-Fischofer-Linaro:pktrefs) ** https://github.com/Linaro/odp/pull/90 ** Patch: https://github.com/Linaro/odp/pull/90.patch ** Base sha: bb0ca193f8c28036fdf8b3d6c8aa5d4f11980e6c ** Merge commit sha: dfef320840f96825686124135a6e509db80b7b82 **/ platform/linux-generic/odp_packet.c | 96 +++++++++++++++++++++++++++++-------- 1 file changed, 77 insertions(+), 19 deletions(-) diff --git a/platform/linux-generic/odp_packet.c b/platform/linux-generic/odp_packet.c index 4dd84e26..5d4be44e 100644 --- a/platform/linux-generic/odp_packet.c +++ b/platform/linux-generic/odp_packet.c @@ -966,6 +966,13 @@ int odp_packet_extend_head(odp_packet_t *pkt, uint32_t len, segs = pkt_hdr->buf_hdr.segcount; if (odp_unlikely((segs + num) > CONFIG_PACKET_MAX_SEGS)) { + /* Corner case: fail request if packet has + * references since we cannot shuffle segments + * since another thread may be accessing them + * concurrently */ + if (packet_ref_count(pkt_hdr) > 1) + return -1; + /* Cannot directly add new segments */ odp_packet_hdr_t *new_hdr; int new_segs = 0; @@ -1043,11 +1050,16 @@ int odp_packet_extend_head(odp_packet_t *pkt, uint32_t len, push_head(pkt_hdr, len); } - if (data_ptr) - *data_ptr = packet_data(pkt_hdr); + if (data_ptr || seg_len) { + uint32_t seg_ln = 0; + void *data = packet_map(pkt_hdr, 0, &seg_ln, NULL); - if (seg_len) - *seg_len = packet_first_seg_len(pkt_hdr); + if (data_ptr) + *data_ptr = data; + + if (seg_len) + *seg_len = seg_ln; + } return ret; } @@ -1068,15 +1080,35 @@ void *odp_packet_pull_head(odp_packet_t pkt, uint32_t len) int odp_packet_trunc_head(odp_packet_t *pkt, uint32_t len, void **data_ptr, uint32_t *seg_len_out) { - odp_packet_hdr_t *pkt_hdr = packet_hdr(*pkt); + odp_packet_hdr_t *pkt_hdr = packet_hdr(*pkt), *nxt_hdr; uint32_t seg_len = packet_first_seg_len(pkt_hdr); + int ret = 0; - if (len > pkt_hdr->frame_len) + if (len > packet_len(pkt_hdr)) return -1; - if (len < seg_len) { + ODP_ASSERT(len <= odp_packet_unshared_len(*pkt)); + + /* Special processing for references */ + while (len >= pkt_hdr->frame_len && pkt_hdr->ref_hdr) { + ODP_ASSERT(packet_ref_count(pkt_hdr) == 1); + nxt_hdr = pkt_hdr->ref_hdr; + len -= pkt_hdr->frame_len; + len += pkt_hdr->ref_offset + + (nxt_hdr->frame_len - pkt_hdr->ref_len); + pkt_hdr->ref_hdr = NULL; + packet_free(pkt_hdr); + pkt_hdr = nxt_hdr; + seg_len = packet_first_seg_len(pkt_hdr); + *pkt = packet_handle(pkt_hdr); + ret = 1; + } + + if (CONFIG_PACKET_MAX_SEGS == 1 || + len < seg_len || + pkt_hdr->buf_hdr.segcount == 1) { pull_head(pkt_hdr, len); - } else if (CONFIG_PACKET_MAX_SEGS != 1) { + } else { int num = 0; uint32_t pull_len = 0; @@ -1091,23 +1123,29 @@ int odp_packet_trunc_head(odp_packet_t *pkt, uint32_t len, *pkt = packet_handle(pkt_hdr); } - if (data_ptr) - *data_ptr = packet_data(pkt_hdr); + if (data_ptr || seg_len_out) { + void *data_head = packet_map(pkt_hdr, 0, &seg_len, NULL); - if (seg_len_out) - *seg_len_out = packet_first_seg_len(pkt_hdr); + if (data_ptr) + *data_ptr = data_head; - return 0; + if (seg_len_out) + *seg_len_out = seg_len; + } + + return ret; } void *odp_packet_push_tail(odp_packet_t pkt, uint32_t len) { - odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt); + odp_packet_hdr_t *pkt_hdr = packet_last_hdr(pkt, NULL); void *old_tail; if (len > pkt_hdr->tailroom) return NULL; + ODP_ASSERT(packet_ref_count(pkt_hdr) == 1); + old_tail = packet_tail(pkt_hdr); push_tail(pkt_hdr, len); @@ -1117,12 +1155,14 @@ void *odp_packet_push_tail(odp_packet_t pkt, uint32_t len) int odp_packet_extend_tail(odp_packet_t *pkt, uint32_t len, void **data_ptr, uint32_t *seg_len_out) { - odp_packet_hdr_t *pkt_hdr = packet_hdr(*pkt); + odp_packet_hdr_t *pkt_hdr = packet_last_hdr(*pkt, NULL); uint32_t frame_len = pkt_hdr->frame_len; uint32_t tailroom = pkt_hdr->tailroom; uint32_t tail_off = frame_len; int ret = 0; + ODP_ASSERT(packet_ref_count(pkt_hdr) == 1); + if (len > tailroom) { pool_t *pool = pkt_hdr->buf_hdr.pool_ptr; int num; @@ -1213,6 +1253,7 @@ void *odp_packet_pull_tail(odp_packet_t pkt, uint32_t len) if (len > packet_last_seg_len(pkt_hdr)) return NULL; + ODP_ASSERT(packet_ref_count(pkt_hdr) == 1); pull_tail(pkt_hdr, len); return packet_tail(pkt_hdr); @@ -1223,17 +1264,34 @@ int odp_packet_trunc_tail(odp_packet_t *pkt, uint32_t len, { int last; uint32_t seg_len; - odp_packet_hdr_t *pkt_hdr = packet_hdr(*pkt); + uint32_t offset; + odp_packet_hdr_t *first_hdr = packet_hdr(*pkt); + odp_packet_hdr_t *pkt_hdr, *prev_hdr; - if (len > pkt_hdr->frame_len) + if (len > packet_len(first_hdr)) return -1; + pkt_hdr = packet_last_hdr(*pkt, &offset); + + /* Special processing for references */ + while (len >= pkt_hdr->frame_len - offset && first_hdr->ref_hdr) { + len -= (pkt_hdr->frame_len - offset); + prev_hdr = packet_prev_hdr(first_hdr, pkt_hdr, &offset); + ODP_ASSERT(packet_ref_count(prev_hdr) == 1); + prev_hdr->ref_hdr = NULL; + packet_free(pkt_hdr); + pkt_hdr = prev_hdr; + } + + ODP_ASSERT(packet_ref_count(pkt_hdr) == 1); last = packet_last_seg(pkt_hdr); seg_len = packet_seg_len(pkt_hdr, last); - if (len < seg_len) { + if (CONFIG_PACKET_MAX_SEGS == 1 || + len < seg_len || + pkt_hdr->buf_hdr.segcount == 1) { pull_tail(pkt_hdr, len); - } else if (CONFIG_PACKET_MAX_SEGS != 1) { + } else { int num = 0; uint32_t pull_len = 0; From patchwork Tue Aug 1 15:00:09 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 109182 Delivered-To: patch@linaro.org Received: by 10.140.101.6 with SMTP id t6csp1616210qge; Tue, 1 Aug 2017 08:09:45 -0700 (PDT) X-Received: by 10.55.126.198 with SMTP id z189mr26501667qkc.299.1501600185537; Tue, 01 Aug 2017 08:09:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501600185; cv=none; d=google.com; s=arc-20160816; b=XR/gb74SVgVIswV4MwcV0vnXNzthw92SFPeEpLX/5Lgqw/DzmwFKAPNCJJLXoV5Q2+ 4cCRv+/FweJVF+FaAvgPXr0D/MMXm+F0CSibfG4i0veNhx7aBZCs3GmYi5o7jYXCD7aM tc1S26ciZfWmt/cjlCt1XpZ2dCcL9hX+dxvZyJjR4j4L8otyo5liZuMylKEjzNHeO2Na UmYYs416s8NKRkDaVZjD7/iZuSYjYFFe6SwYh+krr1rYJ6q9lJ8MG//993tgqKKnS1kT +ecVgZLgrTGHP2hL79MpQFQvU/df0jDndwAYeo3NxvbHPn6Ypt6Jg11GqjXZZCWGcn4v HMKA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=vLKdyg3JouiBY2tso1OJ9r5YZDt6ux2+NPPZOZNjpV0=; b=tMO+TOPWtBhM1eBr5PAJCQwPQKW4niRviibLVKYVzMpa81PrmIXFtx+5mw6KGGBEiq Egt/aadEcIBDFoO3govHHx1oVI58Lpo3LhSgdGATXXxfOR/0SuU3iggPnnkGTxHem0hD zDNhyDYHd19CaQFyIfX1Tdn3erHUVL4Vw1a0sDmqkufQgqIWbo8A7Dr4o1RIku8S52jV 4vjUyvHFginC3a7KM9NzDOXBCVThaZ1RSgsQ0K2AcUs2UfFhLoKmKDPPS4VwfCoBTpN1 Lly0XgSDbJhMrt8ZvbrgeFU9dD3Do8+/5QiA4iPiKCjQhqpe4xXoj4Pg145z2v7tJIV8 CvJQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id v45si25720946qta.33.2017.08.01.08.09.45; Tue, 01 Aug 2017 08:09:45 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 37291658CC; Tue, 1 Aug 2017 15:09:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 7678665C35; Tue, 1 Aug 2017 15:03:12 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id D10786486A; Tue, 1 Aug 2017 15:02:55 +0000 (UTC) Received: from forward5o.cmail.yandex.net (forward5o.cmail.yandex.net [37.9.109.249]) by lists.linaro.org (Postfix) with ESMTPS id B92876486A for ; Tue, 1 Aug 2017 15:00:38 +0000 (UTC) Received: from smtp2j.mail.yandex.net (smtp2j.mail.yandex.net [95.108.130.60]) by forward5o.cmail.yandex.net (Yandex) with ESMTP id A40C12124C for ; Tue, 1 Aug 2017 18:00:37 +0300 (MSK) Received: from smtp2j.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp2j.mail.yandex.net (Yandex) with ESMTP id 9EFB73EC11A4 for ; Tue, 1 Aug 2017 18:00:35 +0300 (MSK) Received: by smtp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id tyzQTD0Qgu-0Zb859lf; Tue, 01 Aug 2017 18:00:35 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) X-Yandex-Suid-Status: 1 0 From: Github ODP bot To: lng-odp@lists.linaro.org Date: Tue, 1 Aug 2017 18:00:09 +0300 Message-Id: <1501599610-28388-10-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> References: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 90 Subject: [lng-odp] [PATCH v3 9/10] linux-generic: ipc: cannot share references across ipc X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Bill Fischofer When attempting to send a reference via IPC, make a copy of the packet as reference sharing is only supported within a single ODP instance. Signed-off-by: Bill Fischofer --- /** Email created from pull request 90 (Bill-Fischofer-Linaro:pktrefs) ** https://github.com/Linaro/odp/pull/90 ** Patch: https://github.com/Linaro/odp/pull/90.patch ** Base sha: bb0ca193f8c28036fdf8b3d6c8aa5d4f11980e6c ** Merge commit sha: dfef320840f96825686124135a6e509db80b7b82 **/ platform/linux-generic/pktio/ipc.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/platform/linux-generic/pktio/ipc.c b/platform/linux-generic/pktio/ipc.c index bc7d7564..8c7db84e 100644 --- a/platform/linux-generic/pktio/ipc.c +++ b/platform/linux-generic/pktio/ipc.c @@ -592,7 +592,9 @@ static int ipc_pktio_send_lockless(pktio_entry_t *pktio_entry, _ipc_free_ring_packets(pktio_entry, pktio_entry->s.ipc.tx.free); - /* Copy packets to shm shared pool if they are in different */ + /* Copy packets to shm shared pool if they are in different + * pool, or if they are references (we can't share across IPC). + */ for (i = 0; i < len; i++) { odp_packet_t pkt = pkt_table[i]; pool_t *ipc_pool = pool_entry_from_hdl(pktio_entry->s.ipc.pool); @@ -602,7 +604,8 @@ static int ipc_pktio_send_lockless(pktio_entry_t *pktio_entry, pkt_hdr = odp_packet_hdr(pkt); pool = pkt_hdr->buf_hdr.pool_ptr; - if (pool->pool_idx != ipc_pool->pool_idx) { + if (pool->pool_idx != ipc_pool->pool_idx || + odp_packet_has_ref(pkt)) { odp_packet_t newpkt; newpkt = odp_packet_copy(pkt, pktio_entry->s.ipc.pool); From patchwork Tue Aug 1 15:00:10 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 109181 Delivered-To: patch@linaro.org Received: by 10.140.101.6 with SMTP id t6csp1615492qge; Tue, 1 Aug 2017 08:09:10 -0700 (PDT) X-Received: by 10.55.191.7 with SMTP id p7mr7614158qkf.288.1501600150770; Tue, 01 Aug 2017 08:09:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1501600150; cv=none; d=google.com; s=arc-20160816; b=hmTP3+jf5YUnXPNIUNzRUWcjILf4luUEAJhX73yMQvKHpKyUgG7dVLw8kaPq+AcR+s Ed9B53GAgswG7b1CNZ8Up4IbOiKXOqJtv8Eyc3Cb9d0i1UguYn+uPCvZ5TsJPDdSyPox U6/WKRIh80u30i4uelLGesBR/2BO/awQWVnj5o/vJydTnKyPIooZ3Ah/wAJ89kBfAuop aE3Z9VHHuD0mCaNOqgMiTj+bh7hbi3fm+FStSatabDSfufGdkrdI1BNnfdQQAxuQC1GH TeKuB8Kr5WP25jwJObByxTOw/lk3hqBhhEDXKcSDOa07Eoo7ayVyHgLXGtssawYMbVln 54WQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=DdqBD5yOrNUsElrDHTgKNiZT04w+Gz56aLpOeAPO/5s=; b=VTCPBjwbJ91H8JS+05o0nfOGYHE50SgAvBuIMlJ/HmCVfeY+6Bet5UKeuvodpItnCs b8hx9Uv70dJKfraf8y2VcPsPg+ZGuPN/hXUEfSGROba7pdDBiVCGHHG562FJB0cKnxK9 P+69VlW6U6Sj9EimS+us5EJWqlTKugpqqlqKdMtFzOhE23El82fB4N4Cm++LywXaM0iI VaHNcIsRjCUkWUDsY01fU3Fby6yJ6DrWoH+0SVNYe53MdJevZGsGXQetBg/vA3jozdWP DuRXuIErzO+4X+PYbuLgopUmRQbL4sC0fWmyBAB1rr2Qgpi9N5tcxcjnDWb2ztxCwU7r ttaQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id w6si18627325qti.517.2017.08.01.08.09.10; Tue, 01 Aug 2017 08:09:10 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 6346765BCE; Tue, 1 Aug 2017 15:09:10 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW,URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 10AAA60C06; Tue, 1 Aug 2017 15:03:04 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id B8CEB646DE; Tue, 1 Aug 2017 15:02:53 +0000 (UTC) Received: from forward2m.cmail.yandex.net (forward2m.cmail.yandex.net [5.255.216.20]) by lists.linaro.org (Postfix) with ESMTPS id C3E4765897 for ; Tue, 1 Aug 2017 15:00:40 +0000 (UTC) Received: from smtp2j.mail.yandex.net (smtp2j.mail.yandex.net [95.108.130.60]) by forward2m.cmail.yandex.net (Yandex) with ESMTP id 718CF210E0 for ; Tue, 1 Aug 2017 18:00:39 +0300 (MSK) Received: from smtp2j.mail.yandex.net (localhost.localdomain [127.0.0.1]) by smtp2j.mail.yandex.net (Yandex) with ESMTP id C77D73EC1192 for ; Tue, 1 Aug 2017 18:00:37 +0300 (MSK) Received: by smtp2j.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id tyzQTD0Qgu-0abOS3pf; Tue, 01 Aug 2017 18:00:36 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) X-Yandex-Suid-Status: 1 0 From: Github ODP bot To: lng-odp@lists.linaro.org Date: Tue, 1 Aug 2017 18:00:10 +0300 Message-Id: <1501599610-28388-11-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> References: <1501599610-28388-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 90 Subject: [lng-odp] [PATCH v3 10/10] linux-generic: packet: enable zero-copy reference creation X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Bill Fischofer Signed-off-by: Bill Fischofer --- /** Email created from pull request 90 (Bill-Fischofer-Linaro:pktrefs) ** https://github.com/Linaro/odp/pull/90 ** Patch: https://github.com/Linaro/odp/pull/90.patch ** Base sha: bb0ca193f8c28036fdf8b3d6c8aa5d4f11980e6c ** Merge commit sha: dfef320840f96825686124135a6e509db80b7b82 **/ platform/linux-generic/odp_packet.c | 172 +++++++++++++++++++++--------------- 1 file changed, 103 insertions(+), 69 deletions(-) diff --git a/platform/linux-generic/odp_packet.c b/platform/linux-generic/odp_packet.c index 5d4be44e..f18bd4dd 100644 --- a/platform/linux-generic/odp_packet.c +++ b/platform/linux-generic/odp_packet.c @@ -1710,6 +1710,109 @@ int odp_packet_split(odp_packet_t *pkt, uint32_t len, odp_packet_t *tail) } /* + * References + */ + +static inline void packet_ref(odp_packet_hdr_t *pkt_hdr) +{ + uint32_t i; + odp_packet_hdr_t *hdr; + + do { + for (i = 0; i < pkt_hdr->buf_hdr.segcount; i++) { + hdr = pkt_hdr->buf_hdr.seg[i].hdr; + packet_ref_inc(hdr); + } + + pkt_hdr = pkt_hdr->ref_hdr; + } while (pkt_hdr); +} + +static inline odp_packet_t packet_splice(odp_packet_hdr_t *pkt_hdr, + uint32_t offset, + odp_packet_hdr_t *ref_hdr) +{ + /* Catch attempted references to stale handles in debug builds */ + ODP_ASSERT(packet_ref_count(pkt_hdr) > 0); + + /* Splicing is from the last section of src pkt */ + while (ref_hdr->ref_hdr) + ref_hdr = ref_hdr->ref_hdr; + + /* Find section where splice begins */ + while (offset >= pkt_hdr->frame_len && pkt_hdr->ref_hdr) { + offset -= (pkt_hdr->frame_len - pkt_hdr->ref_offset); + offset += (pkt_hdr->ref_hdr->frame_len - pkt_hdr->ref_len); + pkt_hdr = pkt_hdr->ref_hdr; + } + + ref_hdr->ref_hdr = pkt_hdr; + ref_hdr->ref_offset = offset; + ref_hdr->ref_len = pkt_hdr->frame_len; + + if (packet_ref_count(pkt_hdr) == 1 || offset < pkt_hdr->unshared_len) + pkt_hdr->unshared_len = offset; + + packet_ref(pkt_hdr); + return packet_handle(ref_hdr); +} + +odp_packet_t odp_packet_ref_static(odp_packet_t pkt) +{ + odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt); + + packet_ref(pkt_hdr); + pkt_hdr->unshared_len = 0; + return pkt; +} + +odp_packet_t odp_packet_ref(odp_packet_t pkt, uint32_t offset) +{ + odp_packet_t hdr; + odp_packet_hdr_t *pkt_hdr; + + if (pkt == ODP_PACKET_INVALID) + return ODP_PACKET_INVALID; + + pkt_hdr = packet_hdr(pkt); + if (offset >= packet_len(pkt_hdr)) + return ODP_PACKET_INVALID; + + hdr = odp_packet_alloc(odp_packet_pool(pkt), 0); + + if (hdr == ODP_PACKET_INVALID) + return ODP_PACKET_INVALID; + + return packet_splice(pkt_hdr, offset, packet_hdr(hdr)); +} + +odp_packet_t odp_packet_ref_pkt(odp_packet_t pkt, uint32_t offset, + odp_packet_t hdr) +{ + odp_packet_hdr_t *pkt_hdr; + + if (pkt == ODP_PACKET_INVALID || + hdr == ODP_PACKET_INVALID || + pkt == hdr) + return ODP_PACKET_INVALID; + + ODP_ASSERT(odp_packet_has_ref(hdr) == 0); + + pkt_hdr = packet_hdr(pkt); + if (offset >= packet_len(pkt_hdr)) + return ODP_PACKET_INVALID; + + return packet_splice(pkt_hdr, offset, packet_hdr(hdr)); +} + +int odp_packet_has_ref(odp_packet_t pkt) +{ + odp_packet_hdr_t *pkt_hdr = packet_hdr(pkt); + + return pkt_hdr->ref_hdr != NULL || packet_ref_count(pkt_hdr) > 1; +} + +/* * * Copy * ******************************************************** @@ -2353,75 +2456,6 @@ uint64_t odp_packet_seg_to_u64(odp_packet_seg_t hdl) return _odp_pri(hdl); } -odp_packet_t odp_packet_ref_static(odp_packet_t pkt) -{ - return odp_packet_copy(pkt, odp_packet_pool(pkt)); -} - -odp_packet_t odp_packet_ref(odp_packet_t pkt, uint32_t offset) -{ - odp_packet_t new; - int ret; - - new = odp_packet_copy(pkt, odp_packet_pool(pkt)); - - if (new == ODP_PACKET_INVALID) { - ODP_ERR("copy failed\n"); - return ODP_PACKET_INVALID; - } - - ret = odp_packet_trunc_head(&new, offset, NULL, NULL); - - if (ret < 0) { - ODP_ERR("trunk_head failed\n"); - odp_packet_free(new); - return ODP_PACKET_INVALID; - } - - return new; -} - -odp_packet_t odp_packet_ref_pkt(odp_packet_t pkt, uint32_t offset, - odp_packet_t hdr) -{ - odp_packet_t new; - int ret; - - new = odp_packet_copy(pkt, odp_packet_pool(pkt)); - - if (new == ODP_PACKET_INVALID) { - ODP_ERR("copy failed\n"); - return ODP_PACKET_INVALID; - } - - if (offset) { - ret = odp_packet_trunc_head(&new, offset, NULL, NULL); - - if (ret < 0) { - ODP_ERR("trunk_head failed\n"); - odp_packet_free(new); - return ODP_PACKET_INVALID; - } - } - - ret = odp_packet_concat(&hdr, new); - - if (ret < 0) { - ODP_ERR("concat failed\n"); - odp_packet_free(new); - return ODP_PACKET_INVALID; - } - - return hdr; -} - -int odp_packet_has_ref(odp_packet_t pkt) -{ - (void)pkt; - - return 0; -} - /* Include non-inlined versions of API functions */ #if ODP_ABI_COMPAT == 1 #include