From patchwork Wed Aug 5 22:55:17 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bill Fischofer X-Patchwork-Id: 51987 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-la0-f71.google.com (mail-la0-f71.google.com [209.85.215.71]) by patches.linaro.org (Postfix) with ESMTPS id B0CEB229FD for ; Wed, 5 Aug 2015 23:03:23 +0000 (UTC) Received: by labkp1 with SMTP id kp1sf20688947lab.0 for ; Wed, 05 Aug 2015 16:03:22 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:delivered-to:from:to:date :message-id:in-reply-to:references:subject:precedence:list-id :list-unsubscribe:list-archive:list-post:list-help:list-subscribe :mime-version:content-type:content-transfer-encoding:errors-to :sender:x-original-sender:x-original-authentication-results :mailing-list; bh=mTjjtt+JhDQ6kr8DAQcXtpoEAy7PRttRQ1Qa0Yt8Q6w=; b=cjbcHNWuVRIXgNX4q81ndmO3I+e+43+CJRLx8HSL4LY/TvkOxnCQaja7GM5q8ZCo0N uD6tcj4SsZ2Cex3vrojugLsdoJxSF9ONIm2EyvO4+PAxjPnBcFtrMLkv5DUK7/x2RrwR BnHe8HCtSjhWzddw18Su9tBWXbSi8bhTY1KCYLuLJ4Iu61xRVD1tCjtqU4YO8HRiJ2X+ En3nmBRAQaeXTOHdapMbD0JMkjnis0+2pnP2QvHQrb8n9LPkKJu00CdTX5pvSN01qHa+ DvxOOB03bFH7NLUz2hQaSUaGqahzbo49wcgtLxlP9cd2RT8kZvMP1YQzcfgJ8/zKkZMI X4mg== X-Gm-Message-State: ALoCoQn2sM4NIA5BoBrGZCDr/eDzHKNgqgWZ2GX6TUR+bYO4hTMdcLCxhr9DmUBgZHj3upfPFMl7 X-Received: by 10.112.99.37 with SMTP id en5mr3446275lbb.7.1438815802407; Wed, 05 Aug 2015 16:03:22 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.207.38 with SMTP id lt6ls254965lac.50.gmail; Wed, 05 Aug 2015 16:03:22 -0700 (PDT) X-Received: by 10.152.203.172 with SMTP id kr12mr3300871lac.71.1438815802003; Wed, 05 Aug 2015 16:03:22 -0700 (PDT) Received: from mail-la0-f49.google.com (mail-la0-f49.google.com. [209.85.215.49]) by mx.google.com with ESMTPS id rc5si3234734lac.166.2015.08.05.16.03.21 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 05 Aug 2015 16:03:21 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.49 as permitted sender) client-ip=209.85.215.49; Received: by labow3 with SMTP id ow3so38428270lab.1 for ; Wed, 05 Aug 2015 16:03:21 -0700 (PDT) X-Received: by 10.152.36.102 with SMTP id p6mr11866593laj.19.1438815801852; Wed, 05 Aug 2015 16:03:21 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.7.198 with SMTP id l6csp615436lba; Wed, 5 Aug 2015 16:03:20 -0700 (PDT) X-Received: by 10.55.18.6 with SMTP id c6mr21299923qkh.96.1438815800140; Wed, 05 Aug 2015 16:03:20 -0700 (PDT) Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id g79si8251483qgd.60.2015.08.05.16.03.19; Wed, 05 Aug 2015 16:03:20 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Received: by lists.linaro.org (Postfix, from userid 109) id 1B0EE6209B; Wed, 5 Aug 2015 23:03:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252.ec2.internal X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from ip-10-142-244-252.ec2.internal (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 09C05620C3; Wed, 5 Aug 2015 22:56:36 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id B9F75620C4; Wed, 5 Aug 2015 22:56:31 +0000 (UTC) Received: from mail-ob0-f169.google.com (mail-ob0-f169.google.com [209.85.214.169]) by lists.linaro.org (Postfix) with ESMTPS id EB3DE620A0 for ; Wed, 5 Aug 2015 22:55:35 +0000 (UTC) Received: by obbfr1 with SMTP id fr1so6465451obb.1 for ; Wed, 05 Aug 2015 15:55:35 -0700 (PDT) X-Received: by 10.182.246.202 with SMTP id xy10mr10243121obc.64.1438815335396; Wed, 05 Aug 2015 15:55:35 -0700 (PDT) Received: from localhost.localdomain (cpe-24-28-70-239.austin.res.rr.com. [24.28.70.239]) by smtp.gmail.com with ESMTPSA id u81sm2658477oie.11.2015.08.05.15.55.34 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 05 Aug 2015 15:55:34 -0700 (PDT) From: Bill Fischofer To: lng-odp@lists.linaro.org Date: Wed, 5 Aug 2015 17:55:17 -0500 Message-Id: <1438815321-12344-10-git-send-email-bill.fischofer@linaro.org> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1438815321-12344-1-git-send-email-bill.fischofer@linaro.org> References: <1438815321-12344-1-git-send-email-bill.fischofer@linaro.org> X-Topics: patch Subject: [lng-odp] [API-NEXT PATCHv8 09/13] linux-generic: queue: add ordered chain enq support X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: bill.fischofer@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.49 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Signed-off-by: Bill Fischofer --- .../linux-generic/include/odp_buffer_internal.h | 4 + platform/linux-generic/odp_queue.c | 151 +++++++++++++++------ 2 files changed, 115 insertions(+), 40 deletions(-) diff --git a/platform/linux-generic/include/odp_buffer_internal.h b/platform/linux-generic/include/odp_buffer_internal.h index c459fce..30f061e 100644 --- a/platform/linux-generic/include/odp_buffer_internal.h +++ b/platform/linux-generic/include/odp_buffer_internal.h @@ -109,6 +109,10 @@ typedef union queue_entry_u queue_entry_t; /* Common buffer header */ typedef struct odp_buffer_hdr_t { struct odp_buffer_hdr_t *next; /* next buf in a list */ + union { /* Multi-use secondary link */ + struct odp_buffer_hdr_t *prev; + struct odp_buffer_hdr_t *link; + }; odp_buffer_bits_t handle; /* handle */ union { uint32_t all; diff --git a/platform/linux-generic/odp_queue.c b/platform/linux-generic/odp_queue.c index a2460e7..e56da71 100644 --- a/platform/linux-generic/odp_queue.c +++ b/platform/linux-generic/odp_queue.c @@ -336,6 +336,7 @@ int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) { int sched = 0; queue_entry_t *origin_qe = buf_hdr->origin_qe; + odp_buffer_hdr_t *buf_tail; /* Need two locks for enq operations from ordered queues */ if (origin_qe) { @@ -404,17 +405,33 @@ int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) /* We're in order, so account for this and proceed with enq */ origin_qe->s.order_out++; + + /* if this element is linked, restore the linked chain */ + buf_tail = buf_hdr->link; + + if (buf_tail) { + buf_hdr->next = buf_tail; + buf_hdr->link = NULL; + + /* find end of the chain */ + while (buf_tail->next) + buf_tail = buf_tail->next; + } else { + buf_tail = buf_hdr; + } + } else { + buf_tail = buf_hdr; } - if (queue->s.head == NULL) { + if (!queue->s.head) { /* Empty queue */ queue->s.head = buf_hdr; - queue->s.tail = buf_hdr; - buf_hdr->next = NULL; + queue->s.tail = buf_tail; + buf_tail->next = NULL; } else { queue->s.tail->next = buf_hdr; - queue->s.tail = buf_hdr; - buf_hdr->next = NULL; + queue->s.tail = buf_tail; + buf_tail->next = NULL; } if (queue->s.status == QUEUE_STATUS_NOTSCHED) { @@ -461,8 +478,23 @@ int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) next_buf = reorder_buf->next; if (odp_likely(reorder_buf->target_qe == queue)) { - reorder_prev = reorder_buf; - reorder_buf = next_buf; + /* promote any chain */ + odp_buffer_hdr_t *reorder_link = + reorder_buf->link; + + if (reorder_link) { + reorder_buf->next = reorder_link; + reorder_buf->link = NULL; + while (reorder_link->next) + reorder_link = + reorder_link->next; + reorder_link->next = next_buf; + reorder_prev = reorder_link; + } else { + reorder_prev = reorder_buf; + } + + reorder_buf = next_buf; release_count++; } else if (!reorder_buf->target_qe) { if (reorder_prev) @@ -523,53 +555,89 @@ int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) int queue_enq_multi(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr[], int num) { int sched = 0; - int i, j; + int i, rc, ret_count = 0; + int ordered_head[num]; + int ordered_count = 0; odp_buffer_hdr_t *tail; + /* Identify ordered chains in the input buffer list */ for (i = 0; i < num; i++) { - /* If any buffer is coming from an ordered queue, enqueue them - * individually since in the general case each might originate - * from a different ordered queue. If any of these fail, the - * return code tells the caller how many succeeded. - */ - if (buf_hdr[i]->origin_qe) { - for (j = 0; j < num; j++) { - if (queue_enq(queue, buf_hdr[j])) - return j; - } - return num; + if (buf_hdr[i]->origin_qe) + ordered_head[ordered_count++] = i; + if (i < num - 1) + buf_hdr[i]->next = buf_hdr[i + 1]; + } + + buf_hdr[num - 1]->next = NULL; + + if (ordered_count) { + if (ordered_head[0] > 0) { + tail = buf_hdr[ordered_head[0] - 1]; + tail->next = NULL; + ret_count = ordered_head[0]; + } else { + tail = NULL; + ret_count = 0; } - buf_hdr[i]->next = i == num - 1 ? NULL : buf_hdr[i + 1]; + } else { + tail = buf_hdr[num - 1]; + ret_count = num; } - tail = buf_hdr[num-1]; + /* Handle regular enq's at start of list */ + if (tail) { + tail->next = NULL; + LOCK(&queue->s.lock); + if (odp_unlikely(queue->s.status < QUEUE_STATUS_READY)) { + UNLOCK(&queue->s.lock); + ODP_ERR("Bad queue status\n"); + return -1; + } - LOCK(&queue->s.lock); - if (odp_unlikely(queue->s.status < QUEUE_STATUS_READY)) { + /* Empty queue */ + if (queue->s.head) + queue->s.tail->next = buf_hdr[0]; + else + queue->s.head = buf_hdr[0]; + + queue->s.tail = tail; + + if (queue->s.status == QUEUE_STATUS_NOTSCHED) { + queue->s.status = QUEUE_STATUS_SCHED; + sched = 1; /* retval: schedule queue */ + } UNLOCK(&queue->s.lock); - ODP_ERR("Bad queue status\n"); - return -1; + + /* Add queue to scheduling */ + if (sched && schedule_queue(queue)) + ODP_ABORT("schedule_queue failed\n"); } - /* Empty queue */ - if (queue->s.head == NULL) - queue->s.head = buf_hdr[0]; - else - queue->s.tail->next = buf_hdr[0]; + /* Handle ordered chains in the list */ + for (i = 0; i < ordered_count; i++) { + int eol = i < ordered_count - 1 ? ordered_head[i + 1] : num; + int list_count = eol - i; - queue->s.tail = tail; + if (i < ordered_count - 1) + buf_hdr[ordered_head[i + 1] - 1]->next = NULL; - if (queue->s.status == QUEUE_STATUS_NOTSCHED) { - queue->s.status = QUEUE_STATUS_SCHED; - sched = 1; /* retval: schedule queue */ - } - UNLOCK(&queue->s.lock); + if (ordered_head[i] < eol - 1) + buf_hdr[ordered_head[i]]->link = + buf_hdr[ordered_head[i] + 1]; + else + buf_hdr[ordered_head[i]]->link = NULL; - /* Add queue to scheduling */ - if (sched && schedule_queue(queue)) - ODP_ABORT("schedule_queue failed\n"); + rc = queue_enq(queue, buf_hdr[ordered_head[i]]); + if (rc < 0) + return ret_count; + + if (rc < list_count) + return ret_count + rc; - return num; /* All events enqueued */ + ret_count += rc; + } + + return ret_count; } int odp_queue_enq_multi(odp_queue_t handle, const odp_event_t ev[], int num) @@ -598,6 +666,9 @@ int odp_queue_enq(odp_queue_t handle, odp_event_t ev) queue = queue_to_qentry(handle); buf_hdr = odp_buf_to_hdr(odp_buffer_from_event(ev)); + /* No chains via this entry */ + buf_hdr->link = NULL; + return queue->s.enqueue(queue, buf_hdr); }