From patchwork Mon Apr 14 11:55:30 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Taras Kondratiuk X-Patchwork-Id: 28320 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ig0-f197.google.com (mail-ig0-f197.google.com [209.85.213.197]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 2EE8820534 for ; Mon, 14 Apr 2014 11:58:13 +0000 (UTC) Received: by mail-ig0-f197.google.com with SMTP id hn18sf12007408igb.8 for ; Mon, 14 Apr 2014 04:58:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:cc:subject:precedence:list-id:list-unsubscribe :list-archive:list-post:list-help:list-subscribe:mime-version :errors-to:sender:x-original-sender :x-original-authentication-results:mailing-list:content-type :content-transfer-encoding; bh=teUibtMAwEUBsy+5mbApTs/NUttdZrGHnzk13k6+njY=; b=RRNr3xmImqVUAeqj47SqvxawqmihVhFj1sOtbkTqSG3EMVbeCRuWianITN9isFEOIu fV8YdV0iMB1FgRlZ7ZLEIMIGoeAOvh3RV3tk9d+qHNeQaZLSNLg4ooxypnKk7wJcgHBa 2tmnsXPPYiPwlqV0fhOTY/giWfVxAWAKUbTFi6+Q0fLHJx1fIged+043FwF1hBx+oE63 823g30D3cTsXDDk1hnBv9XKQzV9mpBjiHAjMqQs02/bNjA81pTjCB8AGJmq1TkoE6Dsr xzf6obPi5ItWI9FAFIOkDe2dxvrr3TSG1VVxMO7WB+XJt5lf5+yaMsW6bTywqti8p6VW Az/Q== X-Gm-Message-State: ALoCoQlKvaU4+o5TyOXJ0TGjU8q0z7BzORK7oig3vKgorf3kwJ2T8slBGqGo6wbXY7mT2EJO/yaP X-Received: by 10.50.51.68 with SMTP id i4mr1921709igo.5.1397476692652; Mon, 14 Apr 2014 04:58:12 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.88.72 with SMTP id s66ls2676627qgd.11.gmail; Mon, 14 Apr 2014 04:58:12 -0700 (PDT) X-Received: by 10.58.111.163 with SMTP id ij3mr1598519veb.26.1397476692463; Mon, 14 Apr 2014 04:58:12 -0700 (PDT) Received: from mail-ve0-f182.google.com (mail-ve0-f182.google.com [209.85.128.182]) by mx.google.com with ESMTPS id sw4si2698874vdc.48.2014.04.14.04.58.12 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 14 Apr 2014 04:58:12 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.128.182 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.128.182; Received: by mail-ve0-f182.google.com with SMTP id jw12so7583155veb.27 for ; Mon, 14 Apr 2014 04:58:12 -0700 (PDT) X-Received: by 10.52.108.164 with SMTP id hl4mr2413550vdb.25.1397476692352; Mon, 14 Apr 2014 04:58:12 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.221.72 with SMTP id ib8csp141335vcb; Mon, 14 Apr 2014 04:58:12 -0700 (PDT) X-Received: by 10.224.30.131 with SMTP id u3mr19217660qac.50.1397476691850; Mon, 14 Apr 2014 04:58:11 -0700 (PDT) Received: from ip-10-141-164-156.ec2.internal (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTPS id k4si6579129qci.67.2014.04.14.04.58.11 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Mon, 14 Apr 2014 04:58:11 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Received: from localhost ([127.0.0.1] helo=ip-10-141-164-156.ec2.internal) by ip-10-141-164-156.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1WZfWV-0001oN-Ex; Mon, 14 Apr 2014 11:58:03 +0000 Received: from mail-lb0-f177.google.com ([209.85.217.177]) by ip-10-141-164-156.ec2.internal with esmtp (Exim 4.76) (envelope-from ) id 1WZfUV-0001kC-Vx for lng-odp@lists.linaro.org; Mon, 14 Apr 2014 11:56:00 +0000 Received: by mail-lb0-f177.google.com with SMTP id z11so5458904lbi.8 for ; Mon, 14 Apr 2014 04:56:01 -0700 (PDT) X-Received: by 10.152.23.9 with SMTP id i9mr837512laf.56.1397476561449; Mon, 14 Apr 2014 04:56:01 -0700 (PDT) Received: from uglx0153363.synapse.com ([195.238.92.128]) by mx.google.com with ESMTPSA id bm3sm14022014lbb.12.2014.04.14.04.56.00 for (version=TLSv1.1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 14 Apr 2014 04:56:00 -0700 (PDT) From: Taras Kondratiuk To: lng-odp@lists.linaro.org Date: Mon, 14 Apr 2014 14:55:30 +0300 Message-Id: <1397476530-20816-11-git-send-email-taras.kondratiuk@linaro.org> X-Mailer: git-send-email 1.7.9.5 In-Reply-To: <1397476530-20816-1-git-send-email-taras.kondratiuk@linaro.org> References: <1397476530-20816-1-git-send-email-taras.kondratiuk@linaro.org> Cc: Filip Moerman Subject: [lng-odp] [PATCH 10/10] Keystone2: Add initial Packet IO X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: lng-odp-bounces@lists.linaro.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: taras.kondratiuk@linaro.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.128.182 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Add simple Packet IO implementation. Packet accelerator is not used, so packets are sent directly to Ethernet switch ports. Signed-off-by: Taras Kondratiuk --- .../linux-keystone2/include/odp_buffer_internal.h | 2 - .../include/odp_buffer_pool_internal.h | 1 + .../linux-keystone2/include/odp_packet_internal.h | 15 +- .../include/odp_packet_io_internal.h | 17 +- .../linux-keystone2/include/odp_queue_internal.h | 1 + platform/linux-keystone2/source/odp_buffer.c | 14 +- platform/linux-keystone2/source/odp_buffer_pool.c | 12 +- platform/linux-keystone2/source/odp_init.c | 4 +- platform/linux-keystone2/source/odp_packet.c | 33 +- platform/linux-keystone2/source/odp_packet_io.c | 405 ++++++++------------ platform/linux-keystone2/source/odp_queue.c | 8 +- 11 files changed, 213 insertions(+), 299 deletions(-) diff --git a/platform/linux-keystone2/include/odp_buffer_internal.h b/platform/linux-keystone2/include/odp_buffer_internal.h index 030b107..8f8ab1c 100644 --- a/platform/linux-keystone2/include/odp_buffer_internal.h +++ b/platform/linux-keystone2/include/odp_buffer_internal.h @@ -60,8 +60,6 @@ typedef struct odp_buffer_hdr_t { void *buf_vaddr; uint32_t free_queue; int type; - struct odp_buffer_hdr_t *next; /* next buf in a list */ - odp_buffer_bits_t handle; /* handle */ } odp_buffer_hdr_t; diff --git a/platform/linux-keystone2/include/odp_buffer_pool_internal.h b/platform/linux-keystone2/include/odp_buffer_pool_internal.h index a77331c..6394a8b 100644 --- a/platform/linux-keystone2/include/odp_buffer_pool_internal.h +++ b/platform/linux-keystone2/include/odp_buffer_pool_internal.h @@ -72,6 +72,7 @@ static inline void *get_pool_entry(odp_buffer_pool_t pool_id) { return pool_entry_ptr[pool_id]; } +uint32_t _odp_pool_get_free_queue(odp_buffer_pool_t pool_id); #ifdef __cplusplus } diff --git a/platform/linux-keystone2/include/odp_packet_internal.h b/platform/linux-keystone2/include/odp_packet_internal.h index 792fc7c..2e2b3eb 100644 --- a/platform/linux-keystone2/include/odp_packet_internal.h +++ b/platform/linux-keystone2/include/odp_packet_internal.h @@ -110,19 +110,11 @@ typedef struct { uint32_t l3_offset; /**< offset to L3 hdr, e.g. IPv4, IPv6 */ uint32_t l4_offset; /**< offset to L4 hdr (TCP, UDP, SCTP, also ICMP) */ - uint32_t frame_len; - odp_pktio_t input; - uint32_t pad; - uint8_t payload[]; - } odp_packet_hdr_t; -ODP_ASSERT(sizeof(odp_packet_hdr_t) == ODP_OFFSETOF(odp_packet_hdr_t, payload), - ODP_PACKET_HDR_T__SIZE_ERR); -ODP_ASSERT(sizeof(odp_packet_hdr_t) % sizeof(uint64_t) == 0, - ODP_PACKET_HDR_T__SIZE_ERR2); +ODP_ASSERT(sizeof(odp_packet_hdr_t) <= 128, ODP_PACKET_HDR_T_SIZE_ERROR); /** * Return the packet header @@ -132,6 +124,11 @@ static inline odp_packet_hdr_t *odp_packet_hdr(odp_packet_t pkt) return (odp_packet_hdr_t *)odp_buf_to_hdr((odp_buffer_t)pkt); } +static inline odp_packet_hdr_t *odp_bufhdr_to_pkthdr(odp_buffer_hdr_t *hdr) +{ + return (odp_packet_hdr_t *)hdr; +} + /** * Parse packet and set internal metadata */ diff --git a/platform/linux-keystone2/include/odp_packet_io_internal.h b/platform/linux-keystone2/include/odp_packet_io_internal.h index 3ab7fa0..38887eb 100644 --- a/platform/linux-keystone2/include/odp_packet_io_internal.h +++ b/platform/linux-keystone2/include/odp_packet_io_internal.h @@ -24,17 +24,22 @@ extern "C" { #include #endif +#define PKTIO_DEV_MAX_NAME_LEN 10 +struct pktio_device { + const char name[PKTIO_DEV_MAX_NAME_LEN]; + uint32_t tx_hw_queue; + uint32_t rx_channel; + uint32_t rx_flow; + uint32_t port_id; +}; + struct pktio_entry { odp_spinlock_t lock; /**< entry spinlock */ int taken; /**< is entry taken(1) or free(0) */ odp_queue_t inq_default; /**< default input queue, if set */ odp_queue_t outq_default; /**< default out queue */ - odp_pktio_params_t params; /**< pktio parameters */ - pkt_sock_t pkt_sock; /**< using socket API for IO */ - pkt_sock_mmap_t pkt_sock_mmap; /**< using socket mmap API for IO */ -#ifdef ODP_HAVE_NETMAP - pkt_netmap_t pkt_nm; /**< using netmap API for IO */ -#endif + odp_buffer_pool_t in_pool; + struct pktio_device *dev; }; typedef union { diff --git a/platform/linux-keystone2/include/odp_queue_internal.h b/platform/linux-keystone2/include/odp_queue_internal.h index c7c84d6..9a13a00 100644 --- a/platform/linux-keystone2/include/odp_queue_internal.h +++ b/platform/linux-keystone2/include/odp_queue_internal.h @@ -73,6 +73,7 @@ struct queue_entry_s { odp_queue_param_t param; odp_pktio_t pktin; odp_pktio_t pktout; + uint32_t out_port_id; uint32_t hw_queue; char name[ODP_QUEUE_NAME_LEN]; }; diff --git a/platform/linux-keystone2/source/odp_buffer.c b/platform/linux-keystone2/source/odp_buffer.c index 7a50aa2..d4c7cfe 100644 --- a/platform/linux-keystone2/source/odp_buffer.c +++ b/platform/linux-keystone2/source/odp_buffer.c @@ -59,15 +59,16 @@ int odp_buffer_snprint(char *str, size_t n, odp_buffer_t buf) len += snprintf(&str[len], n-len, " buf_paddr 0x%x\n", desc->desc.buffPtr); len += snprintf(&str[len], n-len, + " buf_len_o 0x%x\n", desc->desc.origBufferLen); + len += snprintf(&str[len], n-len, + " buf_len 0x%x\n", desc->desc.buffLen); + len += snprintf(&str[len], n-len, " pool %i\n", odp_buf_to_pool(buf)); len += snprintf(&str[len], n-len, " free_queue %u\n", desc->free_queue); len += snprintf(&str[len], n-len, "\n"); - ti_em_rh_dump_mem(desc, sizeof(*desc), "Descriptor dump"); - ti_em_rh_dump_mem(desc->buf_vaddr, 64, "Buffer start"); - return len; } @@ -77,11 +78,18 @@ void odp_buffer_print(odp_buffer_t buf) int max_len = 512; char str[max_len]; int len; + odp_buffer_hdr_t *desc; len = odp_buffer_snprint(str, max_len-1, buf); + if (!len) + return; str[len] = 0; printf("\n%s\n", str); + + desc = odp_buf_to_hdr(buf); + ti_em_rh_dump_mem(desc, sizeof(*desc), "Descriptor dump"); + ti_em_rh_dump_mem(desc->buf_vaddr, 64, "Buffer start"); } void odp_buffer_copy_scatter(odp_buffer_t buf_dst, odp_buffer_t buf_src) diff --git a/platform/linux-keystone2/source/odp_buffer_pool.c b/platform/linux-keystone2/source/odp_buffer_pool.c index 5170433..6ce02d4 100644 --- a/platform/linux-keystone2/source/odp_buffer_pool.c +++ b/platform/linux-keystone2/source/odp_buffer_pool.c @@ -163,7 +163,8 @@ static int link_bufs(pool_entry_t *pool) (void *)buf_addr.v, (void *)buf_addr.p); ODP_DBG("%s: num_bufs: %u, desc_index: %u\n", __func__, num_bufs, desc_index); - + ODP_DBG("%s: free_queue: %u\n", __func__, pool->s.free_queue); + /* FIXME: Need to define error codes somewhere */ if (desc_index == (uint32_t)-1) { ODP_ERR("Failed to allocate %u descriptors for pool %s\n", @@ -195,10 +196,9 @@ static int link_bufs(pool_entry_t *pool) /* Set defaults in descriptor */ hdr->desc.descInfo = (Cppi_DescType_HOST << 30) | (Cppi_PSLoc_PS_IN_DESC << 22) | - (buf_size & 0xFFFF); + (pool->s.payload_size & 0xFFFF); hdr->desc.packetInfo = (((uint32_t) Cppi_EPIB_EPIB_PRESENT) << 31) | - (0x2 << 16) | (((uint32_t) Cppi_ReturnPolicy_RETURN_BUFFER) << 15) | (pool->s.free_queue & 0x3FFF); hdr->desc.origBuffPtr = buf_addr.p; @@ -308,3 +308,9 @@ void odp_buffer_pool_print(odp_buffer_pool_t pool_id) { (void)pool_id; } + +uint32_t _odp_pool_get_free_queue(odp_buffer_pool_t pool_id) +{ + pool_entry_t *pool = get_pool_entry(pool_id); + return pool->s.free_queue; +} diff --git a/platform/linux-keystone2/source/odp_init.c b/platform/linux-keystone2/source/odp_init.c index 0b36960..f832551 100644 --- a/platform/linux-keystone2/source/odp_init.c +++ b/platform/linux-keystone2/source/odp_init.c @@ -12,7 +12,7 @@ #include #include #include -#include +#include /* * Make region_configs[] global, because hw_config is saved in @@ -49,7 +49,7 @@ static int ti_init_hw_config(void) reg_config = ®ion_configs[TI_EM_RH_PUBLIC]; reg_config->region_idx = TI_ODP_PUBLIC_REGION_IDX; reg_config->desc_size = - ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(odp_buffer_hdr_t)); + ODP_CACHE_LINE_SIZE_ROUNDUP(sizeof(odp_packet_hdr_t)); reg_config->desc_num = TI_ODP_PUBLIC_DESC_NUM; reg_config->desc_base = TI_ODP_PUBLIC_DESC_BASE; reg_config->desc_vbase = TI_ODP_PUBLIC_DESC_VBASE; diff --git a/platform/linux-keystone2/source/odp_packet.c b/platform/linux-keystone2/source/odp_packet.c index f03d849..37a0d46 100644 --- a/platform/linux-keystone2/source/odp_packet.c +++ b/platform/linux-keystone2/source/odp_packet.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include @@ -16,20 +17,13 @@ #include static inline uint8_t parse_ipv4(odp_packet_hdr_t *pkt_hdr, odp_ipv4hdr_t *ipv4, - size_t *offset_out); + size_t *offset_out); static inline uint8_t parse_ipv6(odp_packet_hdr_t *pkt_hdr, odp_ipv6hdr_t *ipv6, - size_t *offset_out); + size_t *offset_out); void odp_packet_init(odp_packet_t pkt) { odp_packet_hdr_t *const pkt_hdr = odp_packet_hdr(pkt); - const size_t start_offset = ODP_FIELD_SIZEOF(odp_packet_hdr_t, buf_hdr); - uint8_t *start; - size_t len; - - start = (uint8_t *)pkt_hdr + start_offset; - len = ODP_OFFSETOF(odp_packet_hdr_t, payload) - start_offset; - memset(start, 0, len); pkt_hdr->l2_offset = (uint32_t) ODP_PACKET_OFFSET_INVALID; pkt_hdr->l3_offset = (uint32_t) ODP_PACKET_OFFSET_INVALID; @@ -48,12 +42,12 @@ odp_buffer_t odp_buffer_from_packet(odp_packet_t pkt) void odp_packet_set_len(odp_packet_t pkt, size_t len) { - odp_packet_hdr(pkt)->frame_len = len; + ti_em_cppi_set_pktlen(&odp_packet_hdr(pkt)->buf_hdr.desc, len); } size_t odp_packet_get_len(odp_packet_t pkt) { - return odp_packet_hdr(pkt)->frame_len; + return ti_em_cppi_get_pktlen(&odp_packet_hdr(pkt)->buf_hdr.desc); } uint8_t *odp_packet_buf_addr(odp_packet_t pkt) @@ -130,8 +124,9 @@ void odp_packet_set_l4_offset(odp_packet_t pkt, size_t offset) /** * Simple packet parser: eth, VLAN, IP, TCP/UDP/ICMP * - * Internal function: caller is resposible for passing only valid packet handles - * , lengths and offsets (usually done&called in packet input). + * Internal function: caller is responsible for passing only + * valid packet handles, lengths and offsets + * (usually done&called in packet input). * * @param pkt Packet handle * @param len Packet length in bytes @@ -150,7 +145,6 @@ void odp_packet_parse(odp_packet_t pkt, size_t len, size_t frame_offset) pkt_hdr->input_flags.eth = 1; pkt_hdr->frame_offset = frame_offset; - pkt_hdr->frame_len = len; if (odp_unlikely(len < ODP_ETH_LEN_MIN)) { pkt_hdr->error_flags.frame_len = 1; @@ -159,6 +153,9 @@ void odp_packet_parse(odp_packet_t pkt, size_t len, size_t frame_offset) pkt_hdr->input_flags.jumbo = 1; } + len -= 4; /* Crop L2 CRC */ + ti_em_cppi_set_pktlen(&pkt_hdr->buf_hdr.desc, len); + /* Assume valid L2 header, no CRC/FCS check in SW */ pkt_hdr->input_flags.l2 = 1; pkt_hdr->l2_offset = frame_offset; @@ -236,7 +233,7 @@ void odp_packet_parse(odp_packet_t pkt, size_t len, size_t frame_offset) } static inline uint8_t parse_ipv4(odp_packet_hdr_t *pkt_hdr, odp_ipv4hdr_t *ipv4, - size_t *offset_out) + size_t *offset_out) { uint8_t ihl; uint16_t frag_offset; @@ -276,7 +273,7 @@ static inline uint8_t parse_ipv4(odp_packet_hdr_t *pkt_hdr, odp_ipv4hdr_t *ipv4, } static inline uint8_t parse_ipv6(odp_packet_hdr_t *pkt_hdr, odp_ipv6hdr_t *ipv6, - size_t *offset_out) + size_t *offset_out) { if (ipv6->next_hdr == ODP_IPPROTO_ESP || ipv6->next_hdr == ODP_IPPROTO_AH) { @@ -321,12 +318,14 @@ void odp_packet_print(odp_packet_t pkt) len += snprintf(&str[len], n-len, " l4_offset %u\n", hdr->l4_offset); len += snprintf(&str[len], n-len, - " frame_len %u\n", hdr->frame_len); + " packet len %u\n", odp_packet_get_len(pkt)); len += snprintf(&str[len], n-len, " input %u\n", hdr->input); str[len] = '\0'; printf("\n%s\n", str); + ti_em_rh_dump_mem(hdr, sizeof(*hdr), "Descriptor dump"); + ti_em_rh_dump_mem(hdr->buf_hdr.buf_vaddr, 64, "Buffer start"); } int odp_packet_copy(odp_packet_t pkt_dst, odp_packet_t pkt_src) diff --git a/platform/linux-keystone2/source/odp_packet_io.c b/platform/linux-keystone2/source/odp_packet_io.c index 23e2e40..7f99852 100644 --- a/platform/linux-keystone2/source/odp_packet_io.c +++ b/platform/linux-keystone2/source/odp_packet_io.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #ifdef ODP_HAVE_NETMAP @@ -35,6 +36,23 @@ typedef struct { static pktio_table_t *pktio_tbl; +struct pktio_device pktio_devs[] = { + /* eth0 is used by Linux kernel */ + /* {.name = "eth0", .tx_hw_queue = 648, .rx_channel = 22, .rx_flow = 22, .port_id = 1}, */ + {.name = "eth1", .tx_hw_queue = 648, .rx_channel = 23, .rx_flow = 23, .port_id = 2}, + {.name = "eth2", .tx_hw_queue = 648, .rx_channel = 24, .rx_flow = 24, .port_id = 3}, + {.name = "eth3", .tx_hw_queue = 648, .rx_channel = 25, .rx_flow = 25, .port_id = 4}, +}; + +static struct pktio_device *_odp_pktio_dev_lookup(const char *name) +{ + int i; + int num = sizeof(pktio_devs)/sizeof(pktio_devs[0]); + for (i = 0; i < num; i++) + if (!strncmp(pktio_devs[i].name, name, PKTIO_DEV_MAX_NAME_LEN)) + return &pktio_devs[i]; + return NULL; +} static pktio_entry_t *get_entry(odp_pktio_t id) { @@ -47,11 +65,9 @@ static pktio_entry_t *get_entry(odp_pktio_t id) int odp_pktio_init_global(void) { - char name[ODP_QUEUE_NAME_LEN]; pktio_entry_t *pktio_entry; - queue_entry_t *queue_entry; - odp_queue_t qid; - int id; + int id, i; + int dev_num = sizeof(pktio_devs)/sizeof(pktio_devs[0]); pktio_tbl = odp_shm_reserve("odp_pktio_entries", sizeof(pktio_table_t), @@ -65,20 +81,13 @@ int odp_pktio_init_global(void) pktio_entry = get_entry(id); odp_spinlock_init(&pktio_entry->s.lock); - - /* Create a default output queue for each pktio resource */ - snprintf(name, sizeof(name), "%i-pktio_outq_default", (int)id); - name[ODP_QUEUE_NAME_LEN-1] = '\0'; - - qid = odp_queue_create(name, ODP_QUEUE_TYPE_PKTOUT, NULL); - if (qid == ODP_QUEUE_INVALID) - return -1; - pktio_entry->s.outq_default = qid; - - queue_entry = queue_to_qentry(qid); - queue_entry->s.pktout = id; } + /* Close all used RX channels */ + for (i = 0; i < dev_num; i++) + ti_em_osal_cppi_rx_channel_close(Cppi_CpDma_PASS_CPDMA, + pktio_devs[i].rx_channel); + return 0; } @@ -112,43 +121,20 @@ static void unlock_entry(pktio_entry_t *entry) odp_spinlock_unlock(&entry->s.lock); } -static void init_pktio_entry(pktio_entry_t *entry, odp_pktio_params_t *params) -{ - set_taken(entry); - entry->s.inq_default = ODP_QUEUE_INVALID; - switch (params->type) { - case ODP_PKTIO_TYPE_SOCKET_BASIC: - case ODP_PKTIO_TYPE_SOCKET_MMSG: - case ODP_PKTIO_TYPE_SOCKET_MMAP: - memset(&entry->s.pkt_sock, 0, sizeof(entry->s.pkt_sock)); - memset(&entry->s.pkt_sock_mmap, 0, - sizeof(entry->s.pkt_sock_mmap)); - break; -#ifdef ODP_HAVE_NETMAP - case ODP_PKTIO_TYPE_NETMAP: - memset(&entry->s.pkt_nm, 0, sizeof(entry->s.pkt_nm)); - break; -#endif - default: - ODP_ERR("Packet I/O type not supported. Please recompile\n"); - break; - } - /* Save pktio parameters, type is the most useful */ - memcpy(&entry->s.params, params, sizeof(*params)); -} - static odp_pktio_t alloc_lock_pktio_entry(odp_pktio_params_t *params) { odp_pktio_t id; pktio_entry_t *entry; int i; - + (void)params; for (i = 0; i < ODP_CONFIG_PKTIO_ENTRIES; ++i) { entry = &pktio_tbl->entries[i]; if (is_free(entry)) { lock_entry(entry); if (is_free(entry)) { - init_pktio_entry(entry, params); + set_taken(entry); + entry->s.inq_default = ODP_QUEUE_INVALID; + entry->s.outq_default = ODP_QUEUE_INVALID; id = i + 1; return id; /* return with entry locked! */ } @@ -176,28 +162,16 @@ odp_pktio_t odp_pktio_open(char *dev, odp_buffer_pool_t pool, { odp_pktio_t id; pktio_entry_t *pktio_entry; - int res; + char name[ODP_QUEUE_NAME_LEN]; + queue_entry_t *queue_entry; + odp_queue_t qid = ODP_QUEUE_INVALID; if (params == NULL) { ODP_ERR("Invalid pktio params\n"); return ODP_PKTIO_INVALID; } - switch (params->type) { - case ODP_PKTIO_TYPE_SOCKET_BASIC: - case ODP_PKTIO_TYPE_SOCKET_MMSG: - case ODP_PKTIO_TYPE_SOCKET_MMAP: - ODP_DBG("Allocating socket pktio\n"); - break; -#ifdef ODP_HAVE_NETMAP - case ODP_PKTIO_TYPE_NETMAP: - ODP_DBG("Allocating netmap pktio\n"); - break; -#endif - default: - ODP_ERR("Invalid pktio type: %02x\n", params->type); - return ODP_PKTIO_INVALID; - } + ODP_DBG("Allocating HW pktio\n"); id = alloc_lock_pktio_entry(params); if (id == ODP_PKTIO_INVALID) { @@ -208,44 +182,33 @@ odp_pktio_t odp_pktio_open(char *dev, odp_buffer_pool_t pool, pktio_entry = get_entry(id); - switch (params->type) { - case ODP_PKTIO_TYPE_SOCKET_BASIC: - case ODP_PKTIO_TYPE_SOCKET_MMSG: - res = setup_pkt_sock(&pktio_entry->s.pkt_sock, dev, pool); - if (res == -1) { - close_pkt_sock(&pktio_entry->s.pkt_sock); - free_pktio_entry(id); - id = ODP_PKTIO_INVALID; - } - break; - case ODP_PKTIO_TYPE_SOCKET_MMAP: - res = setup_pkt_sock_mmap(&pktio_entry->s.pkt_sock_mmap, dev, - pool, params->sock_params.fanout); - if (res == -1) { - close_pkt_sock_mmap(&pktio_entry->s.pkt_sock_mmap); - free_pktio_entry(id); - id = ODP_PKTIO_INVALID; - } - break; -#ifdef ODP_HAVE_NETMAP - case ODP_PKTIO_TYPE_NETMAP: - - res = setup_pkt_netmap(&pktio_entry->s.pkt_nm, dev, - pool, ¶ms->nm_params); - if (res == -1) { - close_pkt_netmap(&pktio_entry->s.pkt_nm); - free_pktio_entry(id); - id = ODP_PKTIO_INVALID; - } - break; -#endif - default: + /* Create a default output queue for each pktio resource */ + snprintf(name, sizeof(name), "%i-pktio_outq_default", (int)id); + name[ODP_QUEUE_NAME_LEN-1] = '\0'; + + pktio_entry->s.dev = _odp_pktio_dev_lookup(dev); + if (!pktio_entry->s.dev) { + free_pktio_entry(id); + id = ODP_PKTIO_INVALID; + goto unlock; + } + + qid = _odp_queue_create(name, ODP_QUEUE_TYPE_PKTOUT, NULL, + pktio_entry->s.dev->tx_hw_queue); + ODP_DBG("Created queue %u for hw queue %d\n", (uint32_t)qid, + pktio_entry->s.dev->tx_hw_queue); + if (qid == ODP_QUEUE_INVALID) { free_pktio_entry(id); id = ODP_PKTIO_INVALID; - ODP_ERR("This type of I/O is not supported. Please recompile.\n"); - break; + goto unlock; } + pktio_entry->s.in_pool = pool; + pktio_entry->s.outq_default = qid; + queue_entry = queue_to_qentry(qid); + queue_entry->s.pktout = id; + queue_entry->s.out_port_id = pktio_entry->s.dev->port_id; +unlock: unlock_entry(pktio_entry); return id; } @@ -261,24 +224,10 @@ int odp_pktio_close(odp_pktio_t id) lock_entry(entry); if (!is_free(entry)) { - switch (entry->s.params.type) { - case ODP_PKTIO_TYPE_SOCKET_BASIC: - case ODP_PKTIO_TYPE_SOCKET_MMSG: - res = close_pkt_sock(&entry->s.pkt_sock); - break; - case ODP_PKTIO_TYPE_SOCKET_MMAP: - res = close_pkt_sock_mmap(&entry->s.pkt_sock_mmap); - break; -#ifdef ODP_HAVE_NETMAP - case ODP_PKTIO_TYPE_NETMAP: - res = close_pkt_netmap(&entry->s.pkt_nm); - break; -#endif - default: - break; + /* FIXME: Here rx/tx channels should be closed */ res |= free_pktio_entry(id); - } } + unlock_entry(entry); if (res != 0) @@ -300,79 +249,71 @@ odp_pktio_t odp_pktio_get_input(odp_packet_t pkt) int odp_pktio_recv(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) { pktio_entry_t *pktio_entry = get_entry(id); - int pkts; - int i; + unsigned pkts = 0; + odp_buffer_t buf; if (pktio_entry == NULL) return -1; lock_entry(pktio_entry); - switch (pktio_entry->s.params.type) { - case ODP_PKTIO_TYPE_SOCKET_BASIC: - pkts = recv_pkt_sock_basic(&pktio_entry->s.pkt_sock, - pkt_table, len); - break; - case ODP_PKTIO_TYPE_SOCKET_MMSG: - pkts = recv_pkt_sock_mmsg(&pktio_entry->s.pkt_sock, - pkt_table, len); - break; - case ODP_PKTIO_TYPE_SOCKET_MMAP: - pkts = recv_pkt_sock_mmap(&pktio_entry->s.pkt_sock_mmap, - pkt_table, len); - break; -#ifdef ODP_HAVE_NETMAP - case ODP_PKTIO_TYPE_NETMAP: - pkts = recv_pkt_netmap(&pktio_entry->s.pkt_nm, pkt_table, len); - break; -#endif - default: - pkts = -1; - break; - } - unlock_entry(pktio_entry); - if (pkts < 0) - return pkts; + if (pktio_entry->s.inq_default == ODP_QUEUE_INVALID) { + char name[ODP_QUEUE_NAME_LEN]; + odp_queue_param_t qparam; + odp_queue_t inq_def; + /* + * Create a default input queue. + * FIXME: IT is a kind of WA for current ODP API usage. + * It should be revised. + */ + ODP_DBG("Creating default input queue\n"); + qparam.sched.prio = ODP_SCHED_PRIO_DEFAULT; + qparam.sched.sync = ODP_SCHED_SYNC_NONE; + qparam.sched.group = ODP_SCHED_GROUP_DEFAULT; + snprintf(name, sizeof(name), "%i-pktio_inq_default", (int)id); + name[ODP_QUEUE_NAME_LEN-1] = '\0'; + inq_def = odp_queue_create(name, ODP_QUEUE_TYPE_PKTIN, &qparam); + if (inq_def == ODP_QUEUE_INVALID) { + ODP_ERR("pktio queue creation failed\n"); + goto unlock; + } + + if (odp_pktio_inq_setdef(id, inq_def)) { + ODP_ERR("default input-Q setup\n"); + goto unlock; + } + } - for (i = 0; i < pkts; ++i) - odp_pktio_set_input(pkt_table[i], id); + for (pkts = 0; pkts < len; pkts++) { + buf = odp_queue_deq(pktio_entry->s.inq_default); + if (!odp_buffer_is_valid(buf)) + break; + pkt_table[pkts] = odp_packet_from_buffer(buf); + } +unlock: + unlock_entry(pktio_entry); return pkts; } int odp_pktio_send(odp_pktio_t id, odp_packet_t pkt_table[], unsigned len) { pktio_entry_t *pktio_entry = get_entry(id); - int pkts; + unsigned pkts; + int ret; if (pktio_entry == NULL) return -1; lock_entry(pktio_entry); - switch (pktio_entry->s.params.type) { - case ODP_PKTIO_TYPE_SOCKET_BASIC: - pkts = send_pkt_sock_basic(&pktio_entry->s.pkt_sock, - pkt_table, len); - break; - case ODP_PKTIO_TYPE_SOCKET_MMSG: - pkts = send_pkt_sock_mmsg(&pktio_entry->s.pkt_sock, - pkt_table, len); - break; - case ODP_PKTIO_TYPE_SOCKET_MMAP: - pkts = send_pkt_sock_mmap(&pktio_entry->s.pkt_sock_mmap, - pkt_table, len); - break; -#ifdef ODP_HAVE_NETMAP - case ODP_PKTIO_TYPE_NETMAP: - pkts = send_pkt_netmap(&pktio_entry->s.pkt_nm, - pkt_table, len); - break; -#endif - default: - pkts = -1; + + for (pkts = 0; pkts < len; pkts++) { + ret = odp_queue_enq(pktio_entry->s.outq_default, + odp_buffer_from_packet(pkt_table[pkts])); + if (ret) + break; } unlock_entry(pktio_entry); - return pkts; } @@ -387,9 +328,25 @@ int odp_pktio_inq_setdef(odp_pktio_t id, odp_queue_t queue) if (qentry->s.type != ODP_QUEUE_TYPE_PKTIN) return -1; - lock_entry(pktio_entry); pktio_entry->s.inq_default = queue; - unlock_entry(pktio_entry); + { + uint32_t free_queue = + _odp_pool_get_free_queue(pktio_entry->s.in_pool); + ti_em_osal_cppi_rx_channel_close(Cppi_CpDma_PASS_CPDMA, + pktio_entry->s.dev->rx_channel); + ti_em_osal_cppi_rx_flow_open(Cppi_CpDma_PASS_CPDMA, + pktio_entry->s.dev->rx_flow, + qentry->s.hw_queue, + free_queue, + 0); + ti_em_osal_cppi_rx_channel_open(Cppi_CpDma_PASS_CPDMA, + pktio_entry->s.dev->rx_channel); + ODP_DBG("%s: Opened rx flow %u with dest queue: %u and free queue: %u\n", + __func__, + pktio_entry->s.dev->rx_flow, + qentry->s.hw_queue, + free_queue); + } queue_lock(qentry); qentry->s.pktin = id; @@ -426,112 +383,54 @@ odp_queue_t odp_pktio_outq_getdef(odp_pktio_t id) return pktio_entry->s.outq_default; } -int pktout_enqueue(queue_entry_t *qentry, odp_buffer_hdr_t *buf_hdr) +int pktout_enqueue(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr) { - odp_packet_t pkt = odp_packet_from_buffer(buf_hdr->handle.handle); - int len = 1; - int nbr; - - nbr = odp_pktio_send(qentry->s.pktout, &pkt, len); - return (nbr == len ? 0 : -1); -} - -odp_buffer_hdr_t *pktout_dequeue(queue_entry_t *qentry) -{ - (void)qentry; - return NULL; + /* + * Set port number directly in a descriptor. + * TODO: Remove it when PA will be used. + */ + ti_em_cppi_set_psflags(&buf_hdr->desc, queue->s.out_port_id); + return queue_enq(queue, buf_hdr); } -int pktout_enq_multi(queue_entry_t *qentry, odp_buffer_hdr_t *buf_hdr[], - int num) +int pktout_enq_multi(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr[], int num) { - odp_packet_t pkt_tbl[QUEUE_MULTI_MAX]; - int nbr; int i; - - for (i = 0; i < num; ++i) - pkt_tbl[i] = odp_packet_from_buffer(buf_hdr[i]->handle.handle); - - nbr = odp_pktio_send(qentry->s.pktout, pkt_tbl, num); - return (nbr == num ? 0 : -1); + uint32_t port_id = queue->s.out_port_id; + for (i = 0; i < num; i++) + ti_em_cppi_set_psflags(&buf_hdr[i]->desc, port_id); + return queue_enq_multi(queue, buf_hdr, num); } -int pktout_deq_multi(queue_entry_t *qentry, odp_buffer_hdr_t *buf_hdr[], - int num) +static inline void update_in_packet(odp_buffer_hdr_t *buf_hdr, + odp_pktio_t pktin) { - (void)qentry; - (void)buf_hdr; - (void)num; - - return 0; + if (!buf_hdr) + return; + + odp_buffer_t buf = hdr_to_odp_buf(buf_hdr); + odp_packet_t pkt = odp_packet_from_buffer(buf); + odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt); + size_t len = odp_packet_get_len(pkt); + pkt_hdr->input = pktin; + odp_packet_parse(pkt, len, 0); } -int pktin_enqueue(queue_entry_t *qentry, odp_buffer_hdr_t *buf_hdr) -{ - /* Use default action */ - return queue_enq(qentry, buf_hdr); -} - -odp_buffer_hdr_t *pktin_dequeue(queue_entry_t *qentry) +odp_buffer_hdr_t *pktin_dequeue(queue_entry_t *queue) { odp_buffer_hdr_t *buf_hdr; + buf_hdr = queue_deq(queue); - buf_hdr = queue_deq(qentry); - - if (buf_hdr == NULL) { - odp_packet_t pkt; - odp_buffer_t buf; - odp_packet_t pkt_tbl[QUEUE_MULTI_MAX]; - odp_buffer_hdr_t *tmp_hdr_tbl[QUEUE_MULTI_MAX]; - int pkts, i, j; - - pkts = odp_pktio_recv(qentry->s.pktin, pkt_tbl, - QUEUE_MULTI_MAX); - - if (pkts > 0) { - pkt = pkt_tbl[0]; - buf = odp_buffer_from_packet(pkt); - buf_hdr = odp_buf_to_hdr(buf); - - for (i = 1, j = 0; i < pkts; ++i) { - buf = odp_buffer_from_packet(pkt_tbl[i]); - tmp_hdr_tbl[j++] = odp_buf_to_hdr(buf); - } - queue_enq_multi(qentry, tmp_hdr_tbl, j); - } - } - + update_in_packet(buf_hdr, queue->s.pktin); return buf_hdr; } -int pktin_enq_multi(queue_entry_t *qentry, odp_buffer_hdr_t *buf_hdr[], int num) +int pktin_deq_multi(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr[], int num) { - /* Use default action */ - return queue_enq_multi(qentry, buf_hdr, num); -} - -int pktin_deq_multi(queue_entry_t *qentry, odp_buffer_hdr_t *buf_hdr[], int num) -{ - int nbr; - - nbr = queue_deq_multi(qentry, buf_hdr, num); - - if (nbr < num) { - odp_packet_t pkt_tbl[QUEUE_MULTI_MAX]; - odp_buffer_hdr_t *tmp_hdr_tbl[QUEUE_MULTI_MAX]; - odp_buffer_t buf; - int pkts, i; - - pkts = odp_pktio_recv(qentry->s.pktin, pkt_tbl, - QUEUE_MULTI_MAX); - if (pkts > 0) { - for (i = 0; i < pkts; ++i) { - buf = odp_buffer_from_packet(pkt_tbl[i]); - tmp_hdr_tbl[i] = odp_buf_to_hdr(buf); - } - queue_enq_multi(qentry, tmp_hdr_tbl, pkts); - } - } + int i; + num = queue_deq_multi(queue, buf_hdr, num); - return nbr; + for (i = 0; i < num; i++) + update_in_packet(buf_hdr[i], queue->s.pktin); + return num; } diff --git a/platform/linux-keystone2/source/odp_queue.c b/platform/linux-keystone2/source/odp_queue.c index a8768c2..dac6953 100644 --- a/platform/linux-keystone2/source/odp_queue.c +++ b/platform/linux-keystone2/source/odp_queue.c @@ -66,16 +66,16 @@ static void queue_init(queue_entry_t *queue, const char *name, switch (type) { case ODP_QUEUE_TYPE_PKTIN: - queue->s.enqueue = pktin_enqueue; + queue->s.enqueue = queue_enq; queue->s.dequeue = pktin_dequeue; - queue->s.enqueue_multi = pktin_enq_multi; + queue->s.enqueue_multi = queue_enq_multi; queue->s.dequeue_multi = pktin_deq_multi; break; case ODP_QUEUE_TYPE_PKTOUT: queue->s.enqueue = pktout_enqueue; - queue->s.dequeue = pktout_dequeue; + queue->s.dequeue = queue_deq; queue->s.enqueue_multi = pktout_enq_multi; - queue->s.dequeue_multi = pktout_deq_multi; + queue->s.dequeue_multi = queue_deq_multi; break; default: queue->s.enqueue = queue_enq;