From patchwork Thu Jan 11 10:00:10 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 124181 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp455570qgn; Thu, 11 Jan 2018 02:01:58 -0800 (PST) X-Google-Smtp-Source: ACJfBotq5io2XthWsGJk76AXCyOF3V0gB2KlXJ8c4ARGsBUJ6LZqMvRtmPC0CvGFEuaAIdiFTbYn X-Received: by 10.200.28.40 with SMTP id a37mr517813qtk.280.1515664918003; Thu, 11 Jan 2018 02:01:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1515664917; cv=none; d=google.com; s=arc-20160816; b=B00I/9casZgatIDFNLNIRoMmxiijZKlxTX3Uzkmr5s197bwG6kUFcMlG/SvxdjTgpu Pj/sqHqsT+vnehClAgBnDWARnjFSkSG+uX5trGZJzvBnN4tnlu+f8/vXfVw4varFuz+w zGZIaIJ/NAbOjlo00TyVHoc9/0g2f9kV6Ce3uPsvh+AAp2YwZyee7bAcikkJ41pJi7qD K4nmIQA+jTK8aE20tQzzFsktdENniSD7kdlqNFhd4WFHuDhISwn+9XSaRRdRs9T+YY8o e86356M1xQKcNb7ssIzUe3CGRp7Fnty3iUxSWguLIY8Gy3P7RsOcIRWz0RSsKJerjxQf UVbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to :arc-authentication-results; bh=AlbPLg2m7AuQ1TIq5jWZf0raYEw8XT2IHiEfxmNYYrk=; b=R70QEPb0E9kJExuUvOkWUctrNNqchByXnAUh/GfbNr8lMpRS4Rb9Gu6Lx8b2WdVAPV nlPoTLpsoM6M/CaKi2WE82MVfYqmt8iz0y5biKP4NJBrzNMBsSJiVtWCRiblRjzH/Otd TYaA9KYjHXZYqGjgIongKHwlXA7v7NYdNcs2oKLbjKd29VgNw7jVcF9Aa9++26iVq210 TNvtrQy801Jwj5be9tlJ16Q7kTHpUFLaJ6MrH/OB4ESVvaBcS2LV9SZqKRhn2mxY7WUO Li5xROsk4mXXYgt0k9in/tFNhK8p1dAaESxhIlsJf8IvbB48J26RxMA53e9m2iYIADpV IEUQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (ec2-54-197-127-237.compute-1.amazonaws.com. [54.197.127.237]) by mx.google.com with ESMTP id t73si1228039qka.281.2018.01.11.02.01.57; Thu, 11 Jan 2018 02:01:57 -0800 (PST) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) client-ip=54.197.127.237; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id A678D6170E; Thu, 11 Jan 2018 10:01:57 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 972B76150C; Thu, 11 Jan 2018 10:01:38 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 92A89616E8; Thu, 11 Jan 2018 10:01:26 +0000 (UTC) Received: from forward106o.mail.yandex.net (forward106o.mail.yandex.net [37.140.190.187]) by lists.linaro.org (Postfix) with ESMTPS id 960776170E for ; Thu, 11 Jan 2018 10:00:26 +0000 (UTC) Received: from mxback4g.mail.yandex.net (mxback4g.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:165]) by forward106o.mail.yandex.net (Yandex) with ESMTP id 7C1567838C9 for ; Thu, 11 Jan 2018 13:00:25 +0300 (MSK) Received: from smtp2p.mail.yandex.net (smtp2p.mail.yandex.net [2a02:6b8:0:1472:2741:0:8b6:7]) by mxback4g.mail.yandex.net (nwsmtp/Yandex) with ESMTP id bR4nJbn44Q-0P0aXJxw; Thu, 11 Jan 2018 13:00:25 +0300 Received: by smtp2p.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id 9fEAT2lFwy-0IgOed2x; Thu, 11 Jan 2018 13:00:18 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Github ODP bot To: lng-odp@lists.linaro.org Date: Thu, 11 Jan 2018 13:00:10 +0300 Message-Id: <1515664811-8677-2-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1515664811-8677-1-git-send-email-odpbot@yandex.ru> References: <1515664811-8677-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 384 Subject: [lng-odp] [PATCH CATERPILLAR v3 1/1] linux-gen: mediated devices common code X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Ilias Apalodimas Add APIs to create/destroy mediated devices, manipulate their regions and allocate/free DMA areas accessible by both ODP and HW. Signed-off-by: Ilias Apalodimas Signed-off-by: Mykyta Iziumtsev --- /** Email created from pull request 384 (MykytaI:caterpillar_mdev_common) ** https://github.com/Linaro/odp/pull/384 ** Patch: https://github.com/Linaro/odp/pull/384.patch ** Base sha: 4d17f8ae64aba0e6f24877be30f86ae5880cef7e ** Merge commit sha: e5a5cfc0ff3735f1ba8a8441938fb1af7e1df97b **/ configure.ac | 1 + platform/linux-dpdk/m4/configure.m4 | 1 + platform/linux-generic/Makefile.am | 15 +- platform/linux-generic/m4/configure.m4 | 1 + platform/linux-generic/m4/odp_mdev.m4 | 34 ++ platform/linux-generic/pktio/mdev.c | 593 +++++++++++++++++++++++++++ platform/linux-generic/pktio/mdev.h | 61 +++ platform/linux-generic/pktio/uapi_net_mdev.h | 19 + 8 files changed, 719 insertions(+), 6 deletions(-) create mode 100644 platform/linux-generic/m4/odp_mdev.m4 create mode 100644 platform/linux-generic/pktio/mdev.c create mode 100644 platform/linux-generic/pktio/mdev.h create mode 100644 platform/linux-generic/pktio/uapi_net_mdev.h diff --git a/configure.ac b/configure.ac index d774ae86b..3f5092320 100644 --- a/configure.ac +++ b/configure.ac @@ -410,4 +410,5 @@ AC_MSG_RESULT([ test_helper: ${test_helper} test_example: ${test_example} user_guides: ${user_guides} + mdev: ${mdev_support} ]) diff --git a/platform/linux-dpdk/m4/configure.m4 b/platform/linux-dpdk/m4/configure.m4 index 9d299bf35..8c5e06707 100644 --- a/platform/linux-dpdk/m4/configure.m4 +++ b/platform/linux-dpdk/m4/configure.m4 @@ -80,6 +80,7 @@ AC_SUBST([ATOMIC_LIBS]) # linux-generic pktio at all. And DPDK has its own PCAP support anyway AM_CONDITIONAL([HAVE_PCAP], [false]) AM_CONDITIONAL([netmap_support], [false]) +AM_CONDITIONAL([mdev_support], [false]) AM_CONDITIONAL([PKTIO_DPDK], [false]) m4_include([platform/linux-dpdk/m4/odp_pthread.m4]) ODP_TIMER diff --git a/platform/linux-generic/Makefile.am b/platform/linux-generic/Makefile.am index 8813aea92..42223199f 100644 --- a/platform/linux-generic/Makefile.am +++ b/platform/linux-generic/Makefile.am @@ -177,10 +177,12 @@ noinst_HEADERS = \ include/odp_packet_io_pool.h \ include/odp_packet_io_pool_access.h \ include/odp_packet_io_ring_internal.h \ - pktio/ethtool.h \ pktio/common.h \ - pktio/sysfs.h \ pktio/dpdk.h \ + pktio/ethtool.h \ + pktio/mdev.h \ + pktio/sysfs.h \ + pktio/uapi_net_mdev.h \ include/odp_pktio_ops_ipc.h \ include/odp_pktio_ops_loopback.h \ include/odp_pktio_ops_netmap.h \ @@ -243,18 +245,19 @@ __LIB__libodp_linux_la_SOURCES = \ odp_packet_flags.c \ odp_packet_io.c \ odp_packet_io_pool.c \ + pktio/common.c \ + pktio/dpdk.c \ pktio/ethtool.c \ - pktio/subsystem.c \ pktio/ipc.c \ - pktio/common.c \ pktio/loopback.c \ + pktio/mdev.c \ pktio/netmap.c \ - pktio/dpdk.c \ + pktio/ring.c \ pktio/socket.c \ pktio/socket_mmap.c \ + pktio/subsystem.c \ pktio/sysfs.c \ pktio/tap.c \ - pktio/ring.c \ pool/generic.c \ pool/subsystem.c \ odp_pkt_queue.c \ diff --git a/platform/linux-generic/m4/configure.m4 b/platform/linux-generic/m4/configure.m4 index fc29f9cb3..a0d65a319 100644 --- a/platform/linux-generic/m4/configure.m4 +++ b/platform/linux-generic/m4/configure.m4 @@ -14,6 +14,7 @@ m4_include([platform/linux-generic/m4/odp_modules.m4]) m4_include([platform/linux-generic/m4/odp_netmap.m4]) m4_include([platform/linux-generic/m4/odp_dpdk.m4]) m4_include([platform/linux-generic/m4/odp_schedule.m4]) +m4_include([platform/linux-generic/m4/odp_mdev.m4]) m4_include([platform/linux-generic/m4/performance.m4]) diff --git a/platform/linux-generic/m4/odp_mdev.m4 b/platform/linux-generic/m4/odp_mdev.m4 new file mode 100644 index 000000000..64f9e0969 --- /dev/null +++ b/platform/linux-generic/m4/odp_mdev.m4 @@ -0,0 +1,34 @@ +########################################################################## +# Enable mdev support +########################################################################## +mdev_support=no +AC_ARG_ENABLE([mdev_support], + [ --enable-mdev-support include mediated device drivers support], + [if test x$enableval = xyes; then + mdev_support=yes + fi]) + +########################################################################## +# Save and set temporary compilation flags +########################################################################## +OLD_CPPFLAGS=$CPPFLAGS +CPPFLAGS="$MDEV_CPPFLAGS $CPPFLAGS" + +########################################################################## +# Check for mdev availability +########################################################################## +if test x$mdev_support = xyes +then + AC_DEFINE([ODP_MDEV], [1], + [Define to 1 to enable mediated device drivers support]) + AC_SUBST([MDEV_CPPFLAGS]) +else + mdev_support=no +fi + +########################################################################## +# Restore old saved variables +########################################################################## +CPPFLAGS=$OLD_CPPFLAGS + +AM_CONDITIONAL([mdev_support], [test x$mdev_support = xyes ]) diff --git a/platform/linux-generic/pktio/mdev.c b/platform/linux-generic/pktio/mdev.c new file mode 100644 index 000000000..45ea2d113 --- /dev/null +++ b/platform/linux-generic/pktio/mdev.c @@ -0,0 +1,593 @@ +/* Copyright (c) 2018, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#include "config.h" + +#ifdef ODP_MDEV + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include +#include +#include + +#include + +#include +#include + +/** + * returns a valid VFIO container + * fd must be closed by caller + */ +static int get_container(void) +{ + int container; + int ret; + + /* Create a new container */ + container = open("/dev/vfio/vfio", O_RDWR); + if (container < 0) { + ODP_ERR("Failed to create new VFIO container\n"); + goto out; + } + + ret = ioctl(container, VFIO_GET_API_VERSION); + if (ret != VFIO_API_VERSION) { + ODP_ERR("VFIO API version mismatch: expected %i, got %i\n", + VFIO_API_VERSION, ret); + goto out; + } + + ret = ioctl(container, VFIO_CHECK_EXTENSION, VFIO_TYPE1_IOMMU); + if (!ret) { + ODP_ERR("Container doesn't support VFIO_TYPE1_IOMMU\n"); + goto out; + } + + return container; + +out: + if (container != -1) + close(container); + return -1; +} + +/** + * returns a valid VFIO group + * fd must be closed by caller + */ +static int get_group(int grp_id) +{ + char path[32]; + int group; + int ret; + struct vfio_group_status group_status = { + .argsz = sizeof(group_status) + }; + + snprintf(path, sizeof(path), "/dev/vfio/%d", grp_id); + group = open(path, O_RDWR); + if (group < 0) { + ODP_ERR("Failed to open %s: %s\n", path, strerror(errno)); + goto out; + } + + ret = ioctl(group, VFIO_GROUP_GET_STATUS, &group_status); + if (ret < 0) { + ODP_ERR("Failed to get group status\n"); + goto out; + } + + /* Test the group is viable and available */ + if (!(group_status.flags & VFIO_GROUP_FLAGS_VIABLE)) { + ODP_ERR("Group is not viable\n"); + goto out; + } + + return group; + +out: + if (group != -1) + close(group); + return -1; +} + +static void vfio_find_sparse_mmaps(struct vfio_info_cap_header *hdr, + struct vfio_region_info_cap_sparse_mmap + **sparse) +{ + *sparse = + odp_container_of(hdr, struct vfio_region_info_cap_sparse_mmap, + header); +} + +static struct vfio_info_cap_header * +vfio_get_region_info_cap(struct vfio_region_info *info, __u16 id) +{ + struct vfio_info_cap_header *hdr; + void *ptr = info; + + if (!(info->flags & VFIO_REGION_INFO_FLAG_CAPS)) + return NULL; + + for (hdr = + (struct vfio_info_cap_header *)(void *)((uint8_t *)ptr + + info->cap_offset); + hdr != ptr; + hdr = (struct vfio_info_cap_header *)(void *)((uint8_t *)ptr + + hdr->next)) { + if (hdr->id == id) + return hdr; + } + + return NULL; +} + +static struct vfio_info_cap_header * +vfio_get_cap_info(struct vfio_region_info *region_info, __u16 id) +{ + struct vfio_info_cap_header *caps = NULL; + + caps = vfio_get_region_info_cap(region_info, id); + + return caps; +} + +int vfio_get_region_sparse_mmaps(struct vfio_region_info *region_info, + struct vfio_region_info_cap_sparse_mmap + **sparse) +{ + struct vfio_info_cap_header *caps = NULL; + int ret = -ENOENT; + + if (region_info->flags & VFIO_REGION_INFO_FLAG_CAPS && + region_info->argsz > sizeof(*region_info)) { + caps = vfio_get_cap_info(region_info, + VFIO_REGION_INFO_CAP_SPARSE_MMAP); + if (!caps) + goto out; + vfio_find_sparse_mmaps(caps, sparse); + if (*sparse) { + for (uint32_t i = 0; i < (*sparse)->nr_areas; i++) + ODP_DBG("Sparse region: %d 0x%llx %llu\n", i, + (*sparse)->areas[i].offset, + (*sparse)->areas[i].size); + } + + ret = 0; + } + +out: + return ret; +} + +/** Match capability type + * returns 0 on succcess + */ +int vfio_get_region_cap_type(struct vfio_region_info *region_info, + mdev_region_class_t *class_info) +{ + struct vfio_info_cap_header *caps = NULL; + int ret = 0; + struct vfio_region_info_cap_type *cap_type; + + if (region_info->flags & VFIO_REGION_INFO_FLAG_CAPS && + region_info->argsz > sizeof(*region_info)) { + caps = vfio_get_cap_info(region_info, + VFIO_REGION_INFO_CAP_TYPE); + if (!caps) { + ret = -EINVAL; + goto out; + } + + cap_type = + odp_container_of(caps, struct vfio_region_info_cap_type, + header); + + class_info->type = cap_type->type; + class_info->subtype = cap_type->subtype; + } +out: + return ret; +} + +/** + * Get specific region info + */ +static struct vfio_region_info *vfio_get_region(mdev_device_t *mdev, + __u32 region) +{ + int ret; + struct vfio_region_info *region_info = NULL; + + ODP_DBG("Region:%d\n", region); + region_info = calloc(1, sizeof(*region_info)); + if (!region_info) + goto out; + + region_info->index = region; + region_info->argsz = sizeof(*region_info); + ret = ioctl(mdev->device, VFIO_DEVICE_GET_REGION_INFO, region_info); + if (ret < 0) { + ODP_ERR("Failed to get PCI region info\n"); + goto out; + } + + if (region_info->argsz > sizeof(*region_info)) { + struct vfio_region_info *tmp; + + tmp = realloc(region_info, region_info->argsz); + if (!tmp) + goto out; + + region_info = tmp; + + ODP_DBG("region info %d with extended capabilities size: %u\n", + region, region_info->argsz); + ret = ioctl(mdev->device, VFIO_DEVICE_GET_REGION_INFO, + region_info); + if (ret < 0) { + ODP_ERR("Failed to get PCI region info\n"); + goto out; + } + } + + if (!region_info->size) { + ODP_DBG("region info %d is empty, skipping\n", region); + goto out; + } + + return region_info; + +out: + if (region_info) + free(region_info); + return NULL; +} + +/** + * mmap a VFIO region + */ +void *mdev_region_mmap(mdev_device_t *mdev, uint64_t offset, uint64_t size) +{ + void *addr; + + /* Make sure we're page aligned */ + ODP_ASSERT(offset == ROUNDUP_ALIGN(offset, ODP_PAGE_SIZE)); + ODP_ASSERT(size == ROUNDUP_ALIGN(size, ODP_PAGE_SIZE)); + + if (mdev->mappings_count >= ARRAY_SIZE(mdev->mappings)) + return MAP_FAILED; + + addr = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED, + mdev->device, offset); + if (addr == MAP_FAILED) + return addr; + + mdev->mappings[mdev->mappings_count].addr = addr; + mdev->mappings[mdev->mappings_count].size = size; + mdev->mappings_count++; + + return addr; +} + +int mdev_dma_area_alloc(mdev_device_t *mdev, mdev_dma_area_t *dma_area) +{ + struct vfio_iommu_type1_dma_map req; + void *tmp; + + /* Make sure we're page aligned */ + if (dma_area->size != ROUNDUP_ALIGN(dma_area->size, ODP_PAGE_SIZE)) + return -EINVAL; + + memset(&req, 0, sizeof(req)); + req.argsz = sizeof(req); + + tmp = mmap(NULL, dma_area->size, PROT_READ | PROT_WRITE, + MAP_SHARED | MAP_ANONYMOUS | MAP_NORESERVE | MAP_FIXED, -1, + 0); + if (tmp == MAP_FAILED) { + ODP_ERR("mmap failed\n"); + return -EFAULT; + } + + dma_area->vaddr = (uint64_t)tmp; + + req.vaddr = dma_area->vaddr; + req.size = dma_area->size; + req.flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE; + + if (ioctl(mdev->device, VFIO_IOMMU_MAP_DMA, &req) < 0) { + ODP_ERR("ioctl failed\n"); + return -EFAULT; + } + + dma_area->iova = req.iova; + + ODP_DBG("dma_area alloc: %llx@%llx -> %llx\n", dma_area->size, + dma_area->vaddr, dma_area->iova); + + return 0; +} + +int mdev_dma_area_free(mdev_device_t *mdev, mdev_dma_area_t *dma_area) +{ + struct vfio_iommu_type1_dma_unmap req; + + memset(&req, 0, sizeof(req)); + req.argsz = sizeof(req); + req.iova = dma_area->iova; + req.size = dma_area->size; + req.flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE; + + if (ioctl(mdev->device, VFIO_IOMMU_UNMAP_DMA, &req) < 0) { + ODP_ERR("ioctl failed\n"); + return -EFAULT; + } + + if (munmap((void *)dma_area->vaddr, dma_area->size) < 0) { + ODP_ERR("munmap failed\n"); + return -EFAULT; + } + + ODP_DBG("dma_area free: %llx@%llx -> %llx\n", dma_area->size, + dma_area->vaddr, dma_area->iova); + + return 0; +} + +static char *mdev_basename(char *path) +{ + char *rpath; + + rpath = basename(path); + if (rpath) + return rpath; + + return NULL; +} + +static int mdev_readlink(const char *path, char *link, size_t linksz) +{ + ssize_t len; + + len = readlink(path, link, linksz - 1); + if (len != -1) { + link[len] = '\0'; + return 0; + } + return -1; +} + +/** + * returns group_id or -1 on fail and fills group_uuid + */ +static int mdev_sysfs_discover(const char *mod_name, const char *if_name, + char *uuid, size_t sz) +{ + int ret; + char *driver, *iommu_group; + char sysfs_path[2048], sysfs_link[2048]; + DIR *dir; + struct dirent *dp; + + /* Don't put / on the end of the path */ + snprintf(sysfs_path, sizeof(sysfs_path), + "/sys/class/net/%s/device/driver", if_name); + ret = mdev_readlink(sysfs_path, sysfs_link, sizeof(sysfs_link)); + if (ret) { + ODP_ERR("Can't locate sysfs driver path\n"); + return -1; + } + + driver = mdev_basename(sysfs_link); + if (!driver) { + ODP_ERR("Can't driver in sysfs\n"); + return -1; + } + + if (strcmp(driver, mod_name)) { + ODP_ERR("Invalid driver name\n"); + return -1; + } + + snprintf(sysfs_path, sizeof(sysfs_path), + "/sys/class/net/%s/device/mdev_supported_types/%s-netmdev/devices/", + if_name, driver); + + dir = opendir(sysfs_path); + if (!dir) + return -1; + + /* We assume only one UUID per network interface */ + uuid[0] = '\0'; + while ((dp = readdir(dir)) != NULL) { + if (strcmp(dp->d_name, ".") && strcmp(dp->d_name, "..")) { + strncpy(uuid, dp->d_name, sz); + break; + } + } + closedir(dir); + + if (uuid[0] == '\0') + return -1; + + snprintf(sysfs_path, sizeof(sysfs_path), + "/sys/bus/mdev/devices/%s/iommu_group", uuid); + ret = mdev_readlink(sysfs_path, sysfs_link, sizeof(sysfs_link)); + if (ret) { + ODP_ERR("Can't locate IOMMU sysfs path\n"); + return -1; + } + + iommu_group = mdev_basename(sysfs_link); + if (!iommu_group) { + ODP_ERR("Can't locate iommu group in sysfs\n"); + return -1; + } + ret = atoi(iommu_group); + + return ret; +} + +/** + * Initialize VFIO variables. + * set IOMMU and get device regions + */ +static int vfio_init_dev(int grp, int container, + struct vfio_group_status *grp_status, + struct vfio_iommu_type1_info *iommu_info, + struct vfio_device_info *dev_info, char *grp_uuid) +{ + int device = -1; + int ret; + + ret = ioctl(container, VFIO_CHECK_EXTENSION, VFIO_TYPE1_IOMMU); + if (ret < 0) { + ODP_ERR("Doesn't support the IOMMU driver we want\n"); + goto out; + } + + /* Test the group is viable and available */ + ret = ioctl(grp, VFIO_GROUP_GET_STATUS, grp_status); + if (ret < 0 || !(grp_status->flags & VFIO_GROUP_FLAGS_VIABLE)) { + ODP_ERR("Can't get status\n"); + goto out; + } + + ret = ioctl(grp, VFIO_GROUP_SET_CONTAINER, &container); + if (ret < 0) { + ODP_ERR("Failed to set container\n"); + goto out; + } + + ret = ioctl(container, VFIO_SET_IOMMU, VFIO_TYPE1_IOMMU); + if (ret < 0) { + ODP_ERR("Failed to set IOMMU\n"); + goto out; + } + + ret = ioctl(container, VFIO_IOMMU_GET_INFO, iommu_info); + if (ret < 0) { + ODP_ERR("Failed to get IOMMU info\n"); + goto out; + } + + ODP_DBG("iova_pgsizes bitmask=0x%llx\n", iommu_info->iova_pgsizes); + /* Get a file descriptor for the device */ + device = ioctl(grp, VFIO_GROUP_GET_DEVICE_FD, grp_uuid); + if (device < 0) { + ODP_ERR("Failed to get device FD\n"); + goto out; + } + + /* Test and setup the device */ + ret = ioctl(device, VFIO_DEVICE_GET_INFO, dev_info); + if (ret < 0) { + ODP_ERR("Failed to get device info\n"); + goto out; + } + + ODP_DBG("Device %d Regions: %d, irqs:%d\n", device, + dev_info->num_regions, dev_info->num_irqs); + + return device; + +out: + return -1; +} + +int mdev_device_create(mdev_device_t *mdev, const char *mod_name, + const char *if_name, + mdev_region_info_cb_t region_info_cb) +{ + struct vfio_group_status group_status = { + .argsz = sizeof(group_status) + }; + struct vfio_iommu_type1_info iommu_info = { + .argsz = sizeof(iommu_info) + }; + struct vfio_device_info device_info = { + .argsz = sizeof(device_info) + }; + int ret; + + memset(mdev, 0, sizeof(*mdev)); + mdev->container = -1; + mdev->group = -1; + + strncpy(mdev->if_name, if_name, sizeof(mdev->if_name) - 1); + + mdev->group_id = + mdev_sysfs_discover(mod_name, mdev->if_name, mdev->group_uuid, + sizeof(mdev->group_uuid)); + if (mdev->group_id < 0) + goto fail; + + mdev->container = get_container(); + if (mdev->container < 0) + goto fail; + + mdev->group = get_group(mdev->group_id); + if (mdev->group < 0) + goto fail; + + mdev->device = + vfio_init_dev(mdev->group, mdev->container, &group_status, + &iommu_info, &device_info, mdev->group_uuid); + if (mdev->device < 0) + goto fail; + + ret = -EINVAL; + for (uint32_t region = 0; region < device_info.num_regions; region++) { + struct vfio_region_info *region_info; + + region_info = vfio_get_region(mdev, region); + if (!region_info) + continue; + + ret = region_info_cb(mdev, region_info); + free(region_info); + if (ret < 0) { + ODP_ERR("Region info cb fail on region_info[%u]\n", + region); + return -1; + } + + ret = 0; + } + + return ret; + +fail: + return -1; +} + +void mdev_device_destroy(mdev_device_t *mdev) +{ + if (mdev->group != -1) + close(mdev->group); + if (mdev->container != -1) + close(mdev->container); + + for (uint16_t i = 0; i < mdev->mappings_count; i++) + munmap(mdev->mappings[i].addr, mdev->mappings[i].size); +} + +#endif /* ODP_MDEV */ diff --git a/platform/linux-generic/pktio/mdev.h b/platform/linux-generic/pktio/mdev.h new file mode 100644 index 000000000..5249a6d05 --- /dev/null +++ b/platform/linux-generic/pktio/mdev.h @@ -0,0 +1,61 @@ +/* Copyright (c) 2018, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#ifndef ODP_PKTIO_MDEV_H_ +#define ODP_PKTIO_MDEV_H_ + +#include +#include + +#define NET_MDEV_PREFIX "mdev:" + +typedef struct { + uint16_t type; + uint16_t subtype; +} mdev_region_class_t; + +typedef struct { + uint64_t vaddr; + uint64_t iova; + uint64_t size; +} mdev_dma_area_t; + +typedef struct { + char if_name[IF_NAMESIZE]; /**< Interface name */ + + int container; + int group; + int device; + + int group_id; + char group_uuid[64]; + + struct { + uint8_t *addr; + size_t size; + } mappings[256]; + uint16_t mappings_count; +} mdev_device_t; + +typedef int (*mdev_region_info_cb_t)(mdev_device_t *, + struct vfio_region_info *); + +int mdev_device_create(mdev_device_t *mdev, const char *mod_name, + const char *if_name, mdev_region_info_cb_t cb); +void mdev_device_destroy(mdev_device_t *mdev); + +void *mdev_region_mmap(mdev_device_t *mdev, uint64_t offset, uint64_t size); + +int vfio_get_region_cap_type(struct vfio_region_info *region_info, + mdev_region_class_t *type_info); +int vfio_get_region_sparse_mmaps(struct vfio_region_info *region_info, + struct vfio_region_info_cap_sparse_mmap + **sparse); + +int mdev_dma_area_alloc(mdev_device_t *mdev, mdev_dma_area_t *dma_area); +int mdev_dma_area_free(mdev_device_t *mdev, mdev_dma_area_t *dma_area); + +#endif diff --git a/platform/linux-generic/pktio/uapi_net_mdev.h b/platform/linux-generic/pktio/uapi_net_mdev.h new file mode 100644 index 000000000..d355042b4 --- /dev/null +++ b/platform/linux-generic/pktio/uapi_net_mdev.h @@ -0,0 +1,19 @@ +/* Copyright (c) 2018, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#ifndef _UAPI__LINUX_NET_MDEV_H +#define _UAPI__LINUX_NET_MDEV_H + +#include +#include + +enum net_mdev_types { + VFIO_NET_MDEV_MMIO, + VFIO_NET_MDEV_RX_RING, + VFIO_NET_MDEV_TX_RING, +}; + +#endif /* _UAPI__LINUX_NET_MDEV_H */