From patchwork Fri Aug 7 11:47:12 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bill Fischofer X-Patchwork-Id: 52034 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-wi0-f198.google.com (mail-wi0-f198.google.com [209.85.212.198]) by patches.linaro.org (Postfix) with ESMTPS id 4775E20539 for ; Fri, 7 Aug 2015 11:54:37 +0000 (UTC) Received: by wicul11 with SMTP id ul11sf17040078wic.1 for ; Fri, 07 Aug 2015 04:54:36 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:delivered-to:from:to:date :message-id:in-reply-to:references:subject:precedence:list-id :list-unsubscribe:list-archive:list-post:list-help:list-subscribe :mime-version:content-type:content-transfer-encoding:errors-to :sender:x-original-sender:x-original-authentication-results :mailing-list; bh=Ijv0GBfCt4s9EN27W95lDi9Dspc3QBJndKq6O9F5MEk=; b=OOu0pEvREXeT9m/1NGMuDRE+optXoN+XJTDYRMSpUOSpok+3l24oQ1D8Je7iufEJqF szp0illu30pwjtdOgoNGN4C0Q8VePpq+RCNLs6IxF+q3eqL8JGgHQDFw/8w28NTdWQqU r2f+vX2mWblbs1V3eTAkaBa7rFRXM2CDZ1Iks6S2tITK4f78I3N1RN4uGgV4c/tyY4pn EWpHa5RPOeT6Yb2GYdvaeS9/MdaOFKs20a6fRLIb3IGUbJKQUVfOjs5S0Ljn4SNA3Ge4 sYPuuBRD4mcAHFpMbDX75hQxxFVMQLbGtH0TwkTZ1eBEbXnWaSl8X96Kokcou2YJZUpy sRTQ== X-Gm-Message-State: ALoCoQmBMe46ArfzNNWeaE56HzeArUuGQuEf23giI3xV6srWi3dEU7HlctJkqqHO/wxT5FxUBqWt X-Received: by 10.112.130.200 with SMTP id og8mr2009546lbb.21.1438948476561; Fri, 07 Aug 2015 04:54:36 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.219.34 with SMTP id pl2ls67796lac.84.gmail; Fri, 07 Aug 2015 04:54:36 -0700 (PDT) X-Received: by 10.152.6.102 with SMTP id z6mr6886622laz.95.1438948476412; Fri, 07 Aug 2015 04:54:36 -0700 (PDT) Received: from mail-lb0-f173.google.com (mail-lb0-f173.google.com. [209.85.217.173]) by mx.google.com with ESMTPS id tq1si7253293lbb.154.2015.08.07.04.54.36 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 07 Aug 2015 04:54:36 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.173 as permitted sender) client-ip=209.85.217.173; Received: by lbbpu9 with SMTP id pu9so32028485lbb.3 for ; Fri, 07 Aug 2015 04:54:36 -0700 (PDT) X-Received: by 10.152.36.161 with SMTP id r1mr7061454laj.88.1438948476270; Fri, 07 Aug 2015 04:54:36 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.7.198 with SMTP id l6csp322136lba; Fri, 7 Aug 2015 04:54:34 -0700 (PDT) X-Received: by 10.140.152.203 with SMTP id 194mr13110289qhy.19.1438948474188; Fri, 07 Aug 2015 04:54:34 -0700 (PDT) Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id b110si17543065qgf.8.2015.08.07.04.54.31; Fri, 07 Aug 2015 04:54:34 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Received: by lists.linaro.org (Postfix, from userid 109) id 9EA71620F4; Fri, 7 Aug 2015 11:54:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252.ec2.internal X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H2, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from ip-10-142-244-252.ec2.internal (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 6E0AB6214F; Fri, 7 Aug 2015 11:48:45 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id F2554621F4; Fri, 7 Aug 2015 11:48:40 +0000 (UTC) Received: from mail-ob0-f176.google.com (mail-ob0-f176.google.com [209.85.214.176]) by lists.linaro.org (Postfix) with ESMTPS id DC465620E1 for ; Fri, 7 Aug 2015 11:47:29 +0000 (UTC) Received: by obbfr1 with SMTP id fr1so40390426obb.1 for ; Fri, 07 Aug 2015 04:47:29 -0700 (PDT) X-Received: by 10.60.142.234 with SMTP id rz10mr6199567oeb.4.1438948049378; Fri, 07 Aug 2015 04:47:29 -0700 (PDT) Received: from localhost.localdomain (cpe-24-28-70-239.austin.res.rr.com. [24.28.70.239]) by smtp.gmail.com with ESMTPSA id f128sm338865oig.2.2015.08.07.04.47.28 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 07 Aug 2015 04:47:28 -0700 (PDT) From: Bill Fischofer To: lng-odp@lists.linaro.org Date: Fri, 7 Aug 2015 06:47:12 -0500 Message-Id: <1438948036-31868-10-git-send-email-bill.fischofer@linaro.org> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1438948036-31868-1-git-send-email-bill.fischofer@linaro.org> References: <1438948036-31868-1-git-send-email-bill.fischofer@linaro.org> X-Topics: patch Subject: [lng-odp] [API-NEXT PATCHv9 09/13] linux-generic: queue: add ordered chain enq support X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: bill.fischofer@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.173 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Signed-off-by: Bill Fischofer --- .../linux-generic/include/odp_buffer_internal.h | 4 + platform/linux-generic/odp_queue.c | 145 +++++++++++++++------ 2 files changed, 109 insertions(+), 40 deletions(-) diff --git a/platform/linux-generic/include/odp_buffer_internal.h b/platform/linux-generic/include/odp_buffer_internal.h index c459fce..30f061e 100644 --- a/platform/linux-generic/include/odp_buffer_internal.h +++ b/platform/linux-generic/include/odp_buffer_internal.h @@ -109,6 +109,10 @@ typedef union queue_entry_u queue_entry_t; /* Common buffer header */ typedef struct odp_buffer_hdr_t { struct odp_buffer_hdr_t *next; /* next buf in a list */ + union { /* Multi-use secondary link */ + struct odp_buffer_hdr_t *prev; + struct odp_buffer_hdr_t *link; + }; odp_buffer_bits_t handle; /* handle */ union { uint32_t all; diff --git a/platform/linux-generic/odp_queue.c b/platform/linux-generic/odp_queue.c index a2460e7..f7388c7 100644 --- a/platform/linux-generic/odp_queue.c +++ b/platform/linux-generic/odp_queue.c @@ -336,6 +336,7 @@ int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) { int sched = 0; queue_entry_t *origin_qe = buf_hdr->origin_qe; + odp_buffer_hdr_t *buf_tail; /* Need two locks for enq operations from ordered queues */ if (origin_qe) { @@ -404,17 +405,33 @@ int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) /* We're in order, so account for this and proceed with enq */ origin_qe->s.order_out++; + + /* if this element is linked, restore the linked chain */ + buf_tail = buf_hdr->link; + + if (buf_tail) { + buf_hdr->next = buf_tail; + buf_hdr->link = NULL; + + /* find end of the chain */ + while (buf_tail->next) + buf_tail = buf_tail->next; + } else { + buf_tail = buf_hdr; + } + } else { + buf_tail = buf_hdr; } - if (queue->s.head == NULL) { + if (!queue->s.head) { /* Empty queue */ queue->s.head = buf_hdr; - queue->s.tail = buf_hdr; - buf_hdr->next = NULL; + queue->s.tail = buf_tail; + buf_tail->next = NULL; } else { queue->s.tail->next = buf_hdr; - queue->s.tail = buf_hdr; - buf_hdr->next = NULL; + queue->s.tail = buf_tail; + buf_tail->next = NULL; } if (queue->s.status == QUEUE_STATUS_NOTSCHED) { @@ -461,8 +478,23 @@ int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) next_buf = reorder_buf->next; if (odp_likely(reorder_buf->target_qe == queue)) { - reorder_prev = reorder_buf; - reorder_buf = next_buf; + /* promote any chain */ + odp_buffer_hdr_t *reorder_link = + reorder_buf->link; + + if (reorder_link) { + reorder_buf->next = reorder_link; + reorder_buf->link = NULL; + while (reorder_link->next) + reorder_link = + reorder_link->next; + reorder_link->next = next_buf; + reorder_prev = reorder_link; + } else { + reorder_prev = reorder_buf; + } + + reorder_buf = next_buf; release_count++; } else if (!reorder_buf->target_qe) { if (reorder_prev) @@ -523,53 +555,83 @@ int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) int queue_enq_multi(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr[], int num) { int sched = 0; - int i, j; + int i, rc, ret_count = 0; + int ordered_head[num]; + int ordered_count = 0; odp_buffer_hdr_t *tail; + /* Identify ordered chains in the input buffer list */ for (i = 0; i < num; i++) { - /* If any buffer is coming from an ordered queue, enqueue them - * individually since in the general case each might originate - * from a different ordered queue. If any of these fail, the - * return code tells the caller how many succeeded. - */ - if (buf_hdr[i]->origin_qe) { - for (j = 0; j < num; j++) { - if (queue_enq(queue, buf_hdr[j])) - return j; - } - return num; + if (buf_hdr[i]->origin_qe) + ordered_head[ordered_count++] = i; + + buf_hdr[i]->next = i < num - 1 ? buf_hdr[i + 1] : NULL; + } + + if (ordered_count) { + if (ordered_head[0] > 0) { + tail = buf_hdr[ordered_head[0] - 1]; + tail->next = NULL; + ret_count = ordered_head[0]; + } else { + tail = NULL; + ret_count = 0; } - buf_hdr[i]->next = i == num - 1 ? NULL : buf_hdr[i + 1]; + } else { + tail = buf_hdr[num - 1]; + ret_count = num; } - tail = buf_hdr[num-1]; + /* Handle regular enq's at start of list */ + if (tail) { + LOCK(&queue->s.lock); + if (odp_unlikely(queue->s.status < QUEUE_STATUS_READY)) { + UNLOCK(&queue->s.lock); + ODP_ERR("Bad queue status\n"); + return -1; + } + + /* Handle empty queue */ + if (queue->s.head) + queue->s.tail->next = buf_hdr[0]; + else + queue->s.head = buf_hdr[0]; - LOCK(&queue->s.lock); - if (odp_unlikely(queue->s.status < QUEUE_STATUS_READY)) { + queue->s.tail = tail; + + if (queue->s.status == QUEUE_STATUS_NOTSCHED) { + queue->s.status = QUEUE_STATUS_SCHED; + sched = 1; /* retval: schedule queue */ + } UNLOCK(&queue->s.lock); - ODP_ERR("Bad queue status\n"); - return -1; + + /* Add queue to scheduling */ + if (sched && schedule_queue(queue)) + ODP_ABORT("schedule_queue failed\n"); } - /* Empty queue */ - if (queue->s.head == NULL) - queue->s.head = buf_hdr[0]; - else - queue->s.tail->next = buf_hdr[0]; + /* Handle ordered chains in the list */ + for (i = 0; i < ordered_count; i++) { + int eol = i < ordered_count - 1 ? ordered_head[i + 1] : num; + int list_count = eol - i; - queue->s.tail = tail; + if (i < ordered_count - 1) + buf_hdr[eol - 1]->next = NULL; - if (queue->s.status == QUEUE_STATUS_NOTSCHED) { - queue->s.status = QUEUE_STATUS_SCHED; - sched = 1; /* retval: schedule queue */ - } - UNLOCK(&queue->s.lock); + buf_hdr[ordered_head[i]]->link = + list_count > 1 ? buf_hdr[ordered_head[i] + 1] : NULL; - /* Add queue to scheduling */ - if (sched && schedule_queue(queue)) - ODP_ABORT("schedule_queue failed\n"); + rc = queue_enq(queue, buf_hdr[ordered_head[i]]); + if (rc < 0) + return ret_count; + + if (rc < list_count) + return ret_count + rc; - return num; /* All events enqueued */ + ret_count += rc; + } + + return ret_count; } int odp_queue_enq_multi(odp_queue_t handle, const odp_event_t ev[], int num) @@ -598,6 +660,9 @@ int odp_queue_enq(odp_queue_t handle, odp_event_t ev) queue = queue_to_qentry(handle); buf_hdr = odp_buf_to_hdr(odp_buffer_from_event(ev)); + /* No chains via this entry */ + buf_hdr->link = NULL; + return queue->s.enqueue(queue, buf_hdr); }