From patchwork Mon Nov 30 07:48:19 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaying Liang X-Patchwork-Id: 334949 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80354C83014 for ; Mon, 30 Nov 2020 07:49:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 13B242074A for ; Mon, 30 Nov 2020 07:49:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=xilinx.onmicrosoft.com header.i=@xilinx.onmicrosoft.com header.b="gNxJLrVX" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727245AbgK3Ht1 (ORCPT ); Mon, 30 Nov 2020 02:49:27 -0500 Received: from mail-dm6nam12on2060.outbound.protection.outlook.com ([40.107.243.60]:49889 "EHLO NAM12-DM6-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1725965AbgK3Ht0 (ORCPT ); Mon, 30 Nov 2020 02:49:26 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WGL7tYBTNM71vvOmb/JPNM7czuNCTpep4Oc9SddJ142wL+8XzH2h75v7m430urQp6ODn9ujC3e7sor7dx0srNikL/4gwBvrm9gOSe6Jab3rd8l2K+0zXYuN29dWf/90tnbLDXWj/4ZnFwK8nHo8hjHzIc+mwD1zkrcumZSShISBWNEWMHxgVozrvuCyHtuojNeQAwesusvY15RHnYOorww3T00hG2AGJYIk0EZv8WfJBVVl8JfJPuthj4qS1/EXz0VnwN8QpkHFRlj2R9foZ6hICxplfiA6o3XXMxvE+HDvK+V2EuSsmv+yuFVXKh3h6+3GSf8yRRWeDcJD32rWUMQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MXFSnrzUxbGFNY3WMQ8uNSDD2a4l5nIZ1U0M/z9Vbm0=; b=Mm49VjV88jYgMmlsHQqLVn46CkMzIPAyrnDEVQ4J0Ge+SiUl1Kn2K1c+YFt6xbES9db0RwEKz4Y23PLDOOKZGnR9oXo+/duO15GOQ7GmeRlhdDSLKHtZX6s56Cx/0v/Rl9Bdsc7MSIplfZLuWYfexijk77tUBS9L6A2BsydlOGyiprCBsa6DsV9dchNOo1ZL55mG1TXLhes2qYdTVgeRjzx5WATfglWOIJ1xwkU62xXHXYvyYD0QgTTjuVB7Gsa7tjiQYcQZ4I0U69AWZzUNLXk7ygEDCOvqUBA/Q0oRL6wkIGfjcxkMxamz7NQ+cDRpFUkhZb9roLGSKH/EbBmQbw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.62.198) smtp.rcpttodomain=kernel.org smtp.mailfrom=xilinx.com; dmarc=bestguesspass action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MXFSnrzUxbGFNY3WMQ8uNSDD2a4l5nIZ1U0M/z9Vbm0=; b=gNxJLrVXFBKAk9TxPGH7IPf7Z7LK3bJKsN3yXOh99Y/taNYBsMDHlrZnSvo/a260p3f6Ja8RoWPPJFn1ypfzTo2itUKW6m+hhPVecAdgFH13n6M0njKI/KjO4B1XDigGAYIlMEz/we5iwXHGWCBypj6R0CXDIAAsjNtzR22N+vk= Received: from CY4PR03CA0006.namprd03.prod.outlook.com (2603:10b6:903:33::16) by MW4PR02MB7332.namprd02.prod.outlook.com (2603:10b6:303:79::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3611.20; Mon, 30 Nov 2020 07:48:32 +0000 Received: from CY1NAM02FT037.eop-nam02.prod.protection.outlook.com (2603:10b6:903:33:cafe::2a) by CY4PR03CA0006.outlook.office365.com (2603:10b6:903:33::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3611.20 via Frontend Transport; Mon, 30 Nov 2020 07:48:32 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198) smtp.mailfrom=xilinx.com; kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=bestguesspass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.62.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.62.198; helo=xsj-pvapexch02.xlnx.xilinx.com; Received: from xsj-pvapexch02.xlnx.xilinx.com (149.199.62.198) by CY1NAM02FT037.mail.protection.outlook.com (10.152.75.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.3611.27 via Frontend Transport; Mon, 30 Nov 2020 07:48:32 +0000 Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1913.5; Sun, 29 Nov 2020 23:48:31 -0800 Received: from smtp.xilinx.com (172.19.127.96) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id 15.1.1913.5 via Frontend Transport; Sun, 29 Nov 2020 23:48:31 -0800 Envelope-to: michal.simek@xilinx.com, derek.kiernan@xilinx.com, dragan.cvetic@xilinx.com, rajan.vaja@xilinx.com, tejas.patel@xilinx.com, manish.narani@xilinx.com, ravi.patel@xilinx.com, wendy.liang@xilinx.com, robh+dt@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, sumit.semwal@linaro.org, christian.koenig@amd.com, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org Received: from [172.19.2.167] (port=48528 helo=xsjjliang50.xilinx.com) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1kjdvC-0006Pm-Ur; Sun, 29 Nov 2020 23:48:30 -0800 From: Wendy Liang To: , , , , , , , , , , , CC: , , , , , Wendy Liang Subject: [PATCH v3 3/9] misc: xilinx-ai-engine: Implement AI engine cleanup sequence Date: Sun, 29 Nov 2020 23:48:19 -0800 Message-ID: <1606722505-16194-4-git-send-email-wendy.liang@xilinx.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1606722505-16194-1-git-send-email-wendy.liang@xilinx.com> References: <1606722505-16194-1-git-send-email-wendy.liang@xilinx.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f421ea26-113d-431d-11b7-08d89504559f X-MS-TrafficTypeDiagnostic: MW4PR02MB7332: X-Microsoft-Antispam-PRVS: X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply X-MS-Oob-TLC-OOBClassifiers: OLM:3173; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5SAsY0p3v3q0g/IC/8FAmF04BrusR/2Av1M+6i++MbsNfnsIEGFZEKNxV7zxpkC8CC5gcdG1hp2UcX5XjzMma22AqRt2o0okT+EK4Md7t3heEOqlI1p9Lf3/hhtpAbzWoH4/L36l2RTunVGHYxd3BUsZCjraauFX4eyK5lfs0PixcF7mMWJFJ9+li6lmW+OO2GvrlaExHSKOKmYtWBrynC4yJMQ0QicZz5DdRY1Qa7qyRJnCM9wwPZXvCVckRNpcM8mJPd2LXtfsB5lgqp4jDwEU3fSC5edPHmo9ge7b2/5cixcx4VmCZNLG56z9ipWBtkJNvvb/FNWZW0SrMVvaxn/qEvIVLVECWeTA1xZr99tiKqfw46fB9OtDq4L7UtvbDA0NxX0fImyPH0RSZpZ6a26UdDdkkCtbzI0YH2cszqAsLPRqX4T3NZhOUw0DMhNW X-Forefront-Antispam-Report: CIP:149.199.62.198; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:xsj-pvapexch02.xlnx.xilinx.com; PTR:unknown-62-198.xilinx.com; CAT:NONE; SFS:(4636009)(136003)(376002)(346002)(39850400004)(396003)(46966005)(8676002)(7416002)(107886003)(426003)(26005)(54906003)(2906002)(110136005)(70586007)(186003)(4326008)(36906005)(8936002)(316002)(7696005)(70206006)(2616005)(5660300002)(44832011)(82310400003)(478600001)(7636003)(6666004)(921005)(30864003)(356005)(36756003)(336012)(9786002)(83380400001)(6636002)(47076004)(82740400003)(102446001); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Nov 2020 07:48:32.1131 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f421ea26-113d-431d-11b7-08d89504559f X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.62.198]; Helo=[xsj-pvapexch02.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: CY1NAM02FT037.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR02MB7332 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org When AI engine partition is released, that is if no one is using the AI engine partition, by default, it will cleanup the partition by doing the following: * reset the columns * reset the SHIMs * clear data and program memory * gate all the tiles If user doesn't want the partition is reset when the partition is released, user can set the control flag to indicate not to reset the partition when the user requests the partition. If partition the not to reset partition control flag is set, it will not execute the above cleanup sequence when the partition is released. Signed-off-by: Wendy Liang Reviewed-by: Hyun Kwon --- drivers/misc/xilinx-ai-engine/Makefile | 3 +- drivers/misc/xilinx-ai-engine/ai-engine-aie.c | 92 ++++++++++++++++ drivers/misc/xilinx-ai-engine/ai-engine-dev.c | 2 + drivers/misc/xilinx-ai-engine/ai-engine-internal.h | 34 ++++++ drivers/misc/xilinx-ai-engine/ai-engine-part.c | 7 +- drivers/misc/xilinx-ai-engine/ai-engine-reset.c | 121 +++++++++++++++++++++ include/uapi/linux/xlnx-ai-engine.h | 6 + 7 files changed, 259 insertions(+), 6 deletions(-) create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-reset.c diff --git a/drivers/misc/xilinx-ai-engine/Makefile b/drivers/misc/xilinx-ai-engine/Makefile index 7827a0a..39bec61 100644 --- a/drivers/misc/xilinx-ai-engine/Makefile +++ b/drivers/misc/xilinx-ai-engine/Makefile @@ -8,4 +8,5 @@ obj-$(CONFIG_XILINX_AIE) += xilinx-aie.o xilinx-aie-$(CONFIG_XILINX_AIE) := ai-engine-aie.o \ ai-engine-dev.o \ ai-engine-part.o \ - ai-engine-res.o + ai-engine-res.o \ + ai-engine-reset.o diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-aie.c b/drivers/misc/xilinx-ai-engine/ai-engine-aie.c index 319260f..36127f0 100644 --- a/drivers/misc/xilinx-ai-engine/ai-engine-aie.c +++ b/drivers/misc/xilinx-ai-engine/ai-engine-aie.c @@ -5,6 +5,9 @@ * Copyright (C) 2020 Xilinx, Inc. */ +#include +#include +#include #include #include "ai-engine-internal.h" @@ -24,9 +27,25 @@ #define AIE_SHIMPL_L1INTR_MASK_A_REGOFF 0x00035000U #define AIE_SHIMPL_L1INTR_BLOCK_NORTH_B_REGOFF 0x00035050U #define AIE_SHIMPL_CLKCNTR_REGOFF 0x00036040U +#define AIE_SHIMPL_COLRESET_REGOFF 0x00036048U #define AIE_SHIMPL_RESET_REGOFF 0x0003604cU #define AIE_TILE_CORE_CLKCNTR_REGOFF 0x00036040U +/* + * Register masks + */ +#define AIE_SHIMPL_SHIMRST_MASK 0x1U +#define AIE_SHIMPL_COLRST_MASK 0x1U +#define AIE_SHIMPL_CLKCNTR_COLBUF_MASK 0x1U + +/* + * AI engine SHIM reset ID. + * TODO: it should follow the Linux reset framework. The ID should be in the + * device tree. However, as versal resets is not ready, we hardcode it in the + * driver. + */ +#define VERSAL_PM_RST_AIE_SHIM_ID 0xc10405fU + static const struct aie_tile_regs aie_kernel_regs[] = { /* SHIM AXI MM Config */ {.attribute = AIE_TILE_TYPE_SHIMNOC << AIE_REGS_ATTR_TILE_TYPE_SHIFT, @@ -49,6 +68,12 @@ static const struct aie_tile_regs aie_kernel_regs[] = { .soff = AIE_SHIMPL_L1INTR_MASK_A_REGOFF, .eoff = AIE_SHIMPL_L1INTR_BLOCK_NORTH_B_REGOFF, }, + /* SHIM column reset */ + {.attribute = (AIE_TILE_TYPE_SHIMPL | AIE_TILE_TYPE_SHIMNOC) << + AIE_REGS_ATTR_TILE_TYPE_SHIFT, + .soff = AIE_SHIMPL_COLRESET_REGOFF, + .eoff = AIE_SHIMPL_COLRESET_REGOFF, + }, /* SHIM reset Enable */ {.attribute = (AIE_TILE_TYPE_SHIMPL | AIE_TILE_TYPE_SHIMNOC) << AIE_REGS_ATTR_TILE_TYPE_SHIFT, @@ -68,6 +93,16 @@ static const struct aie_tile_regs aie_kernel_regs[] = { }, }; +static const struct aie_single_reg_field aie_col_rst = { + .mask = AIE_SHIMPL_COLRST_MASK, + .regoff = AIE_SHIMPL_COLRESET_REGOFF, +}; + +static const struct aie_single_reg_field aie_col_clkbuf = { + .mask = AIE_SHIMPL_CLKCNTR_COLBUF_MASK, + .regoff = AIE_SHIMPL_CLKCNTR_REGOFF, +}; + static u32 aie_get_tile_type(struct aie_location *loc) { if (loc->row) @@ -79,8 +114,63 @@ static u32 aie_get_tile_type(struct aie_location *loc) return AIE_TILE_TYPE_SHIMNOC; } +/** + * aie_set_shim_reset() - Set AI engine SHIM reset + * @adev: AI engine device + * @range: range of AI engine tiles + * @assert: true to set reset, false to unset reset + */ +static void aie_set_shim_reset(struct aie_device *adev, + struct aie_range *range, bool assert) +{ + u32 c; + u32 val; + struct aie_location loc; + + val = FIELD_PREP(AIE_SHIMPL_SHIMRST_MASK, (assert ? 1 : 0)); + loc.row = 0; + for (c = range->start.col; c < range->start.col + range->size.col; + c++) { + u32 regoff; + + loc.col = c; + regoff = aie_cal_regoff(adev, loc, AIE_SHIMPL_RESET_REGOFF); + iowrite32(val, adev->base + regoff); + } +} + +static int aie_reset_shim(struct aie_device *adev, struct aie_range *range) +{ + int ret; + + /* Enable shim reset of each column */ + aie_set_shim_reset(adev, range, true); + + /* Assert shim reset of AI engine array */ + ret = zynqmp_pm_reset_assert(VERSAL_PM_RST_AIE_SHIM_ID, + PM_RESET_ACTION_ASSERT); + if (ret < 0) { + dev_err(&adev->dev, "failed to assert SHIM reset.\n"); + return ret; + } + + /* Release shim reset of AI engine array */ + ret = zynqmp_pm_reset_assert(VERSAL_PM_RST_AIE_SHIM_ID, + PM_RESET_ACTION_RELEASE); + if (ret < 0) { + dev_err(&adev->dev, "failed to release SHIM reset.\n"); + return ret; + } + + /* Disable shim reset of each column */ + aie_set_shim_reset(adev, range, false); + + return 0; +} + static const struct aie_tile_operations aie_ops = { .get_tile_type = aie_get_tile_type, + .reset_shim = aie_reset_shim, }; /** @@ -104,6 +194,8 @@ int aie_device_init(struct aie_device *adev) adev->ops = &aie_ops; adev->num_kernel_regs = ARRAY_SIZE(aie_kernel_regs); adev->kernel_regs = aie_kernel_regs; + adev->col_rst = &aie_col_rst; + adev->col_clkbuf = &aie_col_clkbuf; /* Get the columns resource */ /* Get number of columns from AI engine memory resource */ diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-dev.c b/drivers/misc/xilinx-ai-engine/ai-engine-dev.c index 5a56a92..69f7216 100644 --- a/drivers/misc/xilinx-ai-engine/ai-engine-dev.c +++ b/drivers/misc/xilinx-ai-engine/ai-engine-dev.c @@ -212,6 +212,8 @@ struct aie_partition *aie_request_partition(struct aie_device *adev, * exported for user to access. */ apart->status = XAIE_PART_STATUS_INUSE; + apart->cntrflag = req->flag; + mutex_unlock(&apart->mlock); } mutex_unlock(&adev->mlock); diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-internal.h b/drivers/misc/xilinx-ai-engine/ai-engine-internal.h index 6a69946..2acd34f 100644 --- a/drivers/misc/xilinx-ai-engine/ai-engine-internal.h +++ b/drivers/misc/xilinx-ai-engine/ai-engine-internal.h @@ -53,18 +53,30 @@ struct aie_tile_regs { u32 attribute; }; +/** + * struct aie_single_reg_field - AI engine single field register attribute + * @mask: field mask + * @regoff: register offset of the field + */ +struct aie_single_reg_field { + u32 mask; + u32 regoff; +}; + struct aie_device; struct aie_partition; /** * struct aie_tile_operations - AI engine device operations * @get_tile_type: get type of tile based on tile operation + * @reset_shim: reset shim, it will assert and then release SHIM reset * * Different AI engine device version has its own device * operation. */ struct aie_tile_operations { u32 (*get_tile_type)(struct aie_location *loc); + int (*reset_shim)(struct aie_device *adev, struct aie_range *range); }; /** @@ -87,6 +99,8 @@ struct aie_resource { * @res: memory resource of AI engine device * @kernel_regs: array of kernel only registers * @ops: tile operations + * @col_rst: column reset attribute + * @col_clkbuf: column clock buffer attribute * @size: size of the AI engine address space * @array_shift: array address shift * @col_shift: column address shift @@ -105,6 +119,8 @@ struct aie_device { struct resource *res; const struct aie_tile_regs *kernel_regs; const struct aie_tile_operations *ops; + const struct aie_single_reg_field *col_rst; + const struct aie_single_reg_field *col_clkbuf; size_t size; struct aie_resource cols_res; u32 array_shift; @@ -124,6 +140,8 @@ struct aie_device { * @partition_id: partition id. Partition ID is the identifier * of the AI engine partition in the system. * @status: indicate if the partition is in use + * @cntrflag: partition control flag. e.g. whether to reset columns when + * the partition is released */ struct aie_partition { struct list_head node; @@ -133,6 +151,7 @@ struct aie_partition { struct device dev; u32 partition_id; u32 status; + u32 cntrflag; }; extern struct class *aie_class; @@ -168,6 +187,20 @@ extern const struct file_operations aie_part_fops; aie_tile_reg_field_get(aie_tile_reg_mask(adev), 0, regoff)) /** + * aie_get_field_val() - calculate value of an AI engine register field + * @field: a field in a register + * @val: value of the field + * @return: value of a register field + */ +static inline u32 aie_get_field_val(const struct aie_single_reg_field *field, + u32 val) +{ + long long mask = (long long)field->mask & 0x00000000ffffffff; + + return (val << __bf_shf(mask)) & field->mask; +} + +/** * aie_cal_regoff() - calculate register offset to the whole AI engine * device start address * @adev: AI engine device @@ -221,6 +254,7 @@ struct aie_partition *aie_request_partition(struct aie_device *adev, struct aie_partition *of_aie_part_probe(struct aie_device *adev, struct device_node *nc); void aie_part_remove(struct aie_partition *apart); +int aie_part_clean(struct aie_partition *apart); int aie_device_init(struct aie_device *adev); #endif /* AIE_INTERNAL_H */ diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-part.c b/drivers/misc/xilinx-ai-engine/ai-engine-part.c index fc8f9f5..98f125b 100644 --- a/drivers/misc/xilinx-ai-engine/ai-engine-part.c +++ b/drivers/misc/xilinx-ai-engine/ai-engine-part.c @@ -217,14 +217,12 @@ static int aie_part_release(struct inode *inode, struct file *filp) struct aie_partition *apart = filp->private_data; int ret; - /* - * TODO: It will need to reset the SHIM columns and gate the - * tiles of the partition. - */ ret = mutex_lock_interruptible(&apart->mlock); if (ret) return ret; + aie_part_clean(apart); + apart->status = 0; mutex_unlock(&apart->mlock); @@ -413,7 +411,6 @@ static struct aie_partition *aie_create_partition(struct aie_device *adev, put_device(dev); return ERR_PTR(ret); } - list_add_tail(&apart->node, &adev->partitions); mutex_unlock(&adev->mlock); get_device(&adev->dev); diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-reset.c b/drivers/misc/xilinx-ai-engine/ai-engine-reset.c new file mode 100644 index 0000000..fc0262f7 --- /dev/null +++ b/drivers/misc/xilinx-ai-engine/ai-engine-reset.c @@ -0,0 +1,121 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Xilinx AI Engine device driver resets implementation + * + * Copyright (C) 2020 Xilinx, Inc. + */ + +#include +#include + +#include "ai-engine-internal.h" + +/** + * aie_part_set_col_reset() - set AI engine column reset + * @apart: AI engine partition + * @col: column to reset + * @reset: true to assert reset, false to release reset + */ +static void aie_part_set_col_reset(struct aie_partition *apart, u32 col, + bool reset) +{ + struct aie_device *adev = apart->adev; + const struct aie_single_reg_field *col_rst = adev->col_rst; + struct aie_location loc; + u32 regoff, val; + + loc.row = 0; + loc.col = col; + + val = aie_get_field_val(col_rst, (reset ? 1 : 0)); + regoff = aie_cal_regoff(adev, loc, col_rst->regoff); + iowrite32(val, adev->base + regoff); +} + +/** + * aie_part_set_col_clkbuf() - set AI engine column clock buffer + * @apart: AI engine partition + * @col: column to reset + * @enable: true to enable, false to disable + */ +static void aie_part_set_col_clkbuf(struct aie_partition *apart, u32 col, + bool enable) +{ + struct aie_device *adev = apart->adev; + const struct aie_single_reg_field *col_clkbuf = adev->col_clkbuf; + struct aie_location loc; + u32 regoff, val; + + loc.row = 0; + loc.col = col; + + val = aie_get_field_val(col_clkbuf, (enable ? 1 : 0)); + regoff = aie_cal_regoff(adev, loc, col_clkbuf->regoff); + iowrite32(val, adev->base + regoff); +} + +/** + * aie_part_set_cols_reset() - set column reset of every column in a partition + * @apart: AI engine partition + * @reset: bool to assert reset, false to release reset + */ +static void aie_part_set_cols_reset(struct aie_partition *apart, bool reset) +{ + struct aie_range *range = &apart->range; + u32 c; + + for (c = range->start.col; c < range->start.col + range->size.col; + c++) + aie_part_set_col_reset(apart, c, reset); +} + +/** + * aie_part_set_cols_clkbuf() - set column clock buffer of every column in a + * partition + * @apart: AI engine partition + * @enable: true to enable, false to disable + */ +static void aie_part_set_cols_clkbuf(struct aie_partition *apart, bool enable) +{ + struct aie_range *range = &apart->range; + u32 c; + + for (c = range->start.col; c < range->start.col + range->size.col; + c++) + aie_part_set_col_clkbuf(apart, c, enable); +} + +/** + * aie_part_clean() - reset and clear AI engine partition + * @apart: AI engine partition + * @return: 0 for success and negative value for failure + * + * This function will: + * * gate all the columns + * * reset AI engine partition columns + * * reset AI engine shims + * * clear the memories + * * gate all the tiles in a partition. + * + * This function will not validate the partition, the caller will need to + * provide a valid AI engine partition. + */ +int aie_part_clean(struct aie_partition *apart) +{ + struct aie_device *adev = apart->adev; + int ret; + + if (apart->cntrflag & XAIE_PART_NOT_RST_ON_RELEASE) + return 0; + + aie_part_set_cols_clkbuf(apart, false); + aie_part_set_cols_reset(apart, true); + + ret = apart->adev->ops->reset_shim(adev, &apart->range); + if (ret < 0) + return ret; + + aie_part_set_cols_clkbuf(apart, false); + + return 0; +} diff --git a/include/uapi/linux/xlnx-ai-engine.h b/include/uapi/linux/xlnx-ai-engine.h index 2a0239b..0f46151 100644 --- a/include/uapi/linux/xlnx-ai-engine.h +++ b/include/uapi/linux/xlnx-ai-engine.h @@ -15,6 +15,12 @@ /* AI engine partition is in use */ #define XAIE_PART_STATUS_INUSE (1U << 0) +/* + * AI engine partition control flags + */ +/* Not reset when release AI engine partition */ +#define XAIE_PART_NOT_RST_ON_RELEASE 0x00000001U + /** * struct aie_location - AIE location information * @col: column id From patchwork Mon Nov 30 07:48:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaying Liang X-Patchwork-Id: 334948 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC012C83012 for ; Mon, 30 Nov 2020 07:50:10 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8B49E20719 for ; Mon, 30 Nov 2020 07:50:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=xilinx.onmicrosoft.com header.i=@xilinx.onmicrosoft.com header.b="ANi+vfko" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727685AbgK3Htm (ORCPT ); Mon, 30 Nov 2020 02:49:42 -0500 Received: from mail-co1nam11on2083.outbound.protection.outlook.com ([40.107.220.83]:6908 "EHLO NAM11-CO1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727436AbgK3Htl (ORCPT ); Mon, 30 Nov 2020 02:49:41 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kiGf1Q7cXlEtXS+Enxd7riqMaTCPBZJvyFwoD3zuf01/jMf+c2TNhKagiXS2iDfQjRumB0A6ZAtHyyBxwTUAg0XBtiA4gYZOvLc2mm5/boEMJwgPoeZ+y7AYFQPxfVW4zlAcaxVJaiMtU9/VDJoCrmo9Sx/nhyTTzO1VCWpTJI82jKe/5Yy+bH4Xbzdl7z1Je9ksNOu2JuC125NJXJp0w0GVbT//y1MdcKbobQfdC2gKs1ijOPswF9mFMwjH2BAhXMWtYj+xEM6rSCPpRAbYqB1GslUthbCimWP7ObLR78k1fRddZYcdwAjLADQuW45jb1QyV3oipceH5BtRoKvIdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WZYx4q2+VchgbQlZsJuflJIdqL9siZCf7G+SYDm/7oE=; b=nYidTDI6IHzfwgJL/SCRKZHzZVXXzqgfg2JprscyRXf03emsh3tbCR8o96ymOgSqm0+rirMsWEfsQ1zU1wYoEq+bJFKvU0Gu+sb/FgHQppD2y94piZouQcZ0DrUpXUNEYFs8VP2wMzNbNziUJ4U5mlPNUMt90Zv8RlZPQmIcy8EeB9iD7EvaRYnI+KcD3lwkaTOWsdlerklWeKTeTInJRXIAbGUgpD9HPuldiPWzYZCzVxHW/CP6eHaibweWXYMS0pQnUkyklx30mkMiNmZj14NAJGQYUrzqaZYsscOVgYsLJBD8afalMJlyo68ewtBZiPgx1ThtN9VDJri8/YwKsA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.62.198) smtp.rcpttodomain=kernel.org smtp.mailfrom=xilinx.com; dmarc=bestguesspass action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WZYx4q2+VchgbQlZsJuflJIdqL9siZCf7G+SYDm/7oE=; b=ANi+vfkoOMwKYIK6iN5Hux9M635Gkicr634t9wrLZkNIP5KOWqme7mlJasO92TKNO4lC7Dajh7NKFnkoLlP8b13aFSqfLiK86zS7x0C1q6VA0NgBf4ousmMFI8xQOPsM2ynffOL37qKsp78Lr/jLzSqqMQ1BKkpg1CtRMJlU5tU= Received: from CY4PR03CA0022.namprd03.prod.outlook.com (2603:10b6:903:33::32) by SJ0PR02MB7343.namprd02.prod.outlook.com (2603:10b6:a03:29c::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3611.23; Mon, 30 Nov 2020 07:48:43 +0000 Received: from CY1NAM02FT037.eop-nam02.prod.protection.outlook.com (2603:10b6:903:33:cafe::e1) by CY4PR03CA0022.outlook.office365.com (2603:10b6:903:33::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3611.20 via Frontend Transport; Mon, 30 Nov 2020 07:48:43 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198) smtp.mailfrom=xilinx.com; kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=bestguesspass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.62.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.62.198; helo=xsj-pvapexch02.xlnx.xilinx.com; Received: from xsj-pvapexch02.xlnx.xilinx.com (149.199.62.198) by CY1NAM02FT037.mail.protection.outlook.com (10.152.75.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.3611.27 via Frontend Transport; Mon, 30 Nov 2020 07:48:43 +0000 Received: from xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1913.5; Sun, 29 Nov 2020 23:48:31 -0800 Received: from smtp.xilinx.com (172.19.127.96) by xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server id 15.1.1913.5 via Frontend Transport; Sun, 29 Nov 2020 23:48:31 -0800 Envelope-to: michal.simek@xilinx.com, derek.kiernan@xilinx.com, dragan.cvetic@xilinx.com, rajan.vaja@xilinx.com, tejas.patel@xilinx.com, manish.narani@xilinx.com, ravi.patel@xilinx.com, wendy.liang@xilinx.com, robh+dt@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, sumit.semwal@linaro.org, christian.koenig@amd.com, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org Received: from [172.19.2.167] (port=48528 helo=xsjjliang50.xilinx.com) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1kjdvD-0006Pm-0z; Sun, 29 Nov 2020 23:48:31 -0800 From: Wendy Liang To: , , , , , , , , , , , CC: , , , , , Wendy Liang Subject: [PATCH v3 5/9] misc: xilinx-ai-engine: add setting shim dma bd operation Date: Sun, 29 Nov 2020 23:48:21 -0800 Message-ID: <1606722505-16194-6-git-send-email-wendy.liang@xilinx.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1606722505-16194-1-git-send-email-wendy.liang@xilinx.com> References: <1606722505-16194-1-git-send-email-wendy.liang@xilinx.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a55a9d22-267b-40be-3627-08d895045c4f X-MS-TrafficTypeDiagnostic: SJ0PR02MB7343: X-Microsoft-Antispam-PRVS: X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply X-MS-Oob-TLC-OOBClassifiers: OLM:93; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: J4rlNAc2C9WOA27xh6hi2ucyctOGlkvWbWfxibREJHkaP6XJtSAgaIv1PBy1yHiXozAueBo6QM4qdhZso6AjdiCx3c7P0sj09YQuYqdm3pAi0n/1egycZcr7h84ha4328QF+sfiaT8ofeOhFgErnTotSXYT4YJvlPhHKm/Q8qxzXr44t20RhyK8rgZBbCSjdenYrgddaoi3x3/dlsAv5shXBHsvnrssUbTaA3mH4RyKGOSUUfhv8GE5ue6wisezNgL0xi+p6kN8T3kQYXY4pFp3U/PgbCQpvN06Z1lntREfSJ3iUxsoS+vewzQkOKFj7wTH3T1lc+0yiLqaog1mSkaUAcmM7IvSMTwucTpcUYL2A7qZDkY4WQuodoai9hNngjYZjo7V2cnPpRPxFZRPD4zaqG2jzO1chlzo8kFC378QdTpWivxkW3hq8g8E4r0gG X-Forefront-Antispam-Report: CIP:149.199.62.198; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:xsj-pvapexch02.xlnx.xilinx.com; PTR:unknown-62-198.xilinx.com; CAT:NONE; SFS:(4636009)(136003)(346002)(396003)(376002)(39850400004)(46966005)(426003)(107886003)(186003)(30864003)(5660300002)(7636003)(6636002)(6666004)(356005)(336012)(478600001)(2616005)(70586007)(70206006)(2906002)(7696005)(83380400001)(44832011)(9786002)(82310400003)(82740400003)(8676002)(8936002)(110136005)(36756003)(316002)(54906003)(47076004)(921005)(4326008)(26005)(36906005)(7416002)(102446001); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Nov 2020 07:48:43.3398 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a55a9d22-267b-40be-3627-08d895045c4f X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.62.198]; Helo=[xsj-pvapexch02.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: CY1NAM02FT037.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR02MB7343 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add operation to set SHIM DMA buffer descriptor. The following operations are added to set the buffer descriptors: * attach DMA buffer which enables AI engine device to access the DMA buffer memory * detach DMA buffer which disables AI engine device to access the DMA buffer memory * set DMA buffer descriptor, which takes buffer descriptor contents pointer, the dmabuf fd, and the offset to the start of dmabuf as as argument. It validates the dmabuf and the offset and size of the buffer. And then it calculates the DMA address of the buffer and set the buffer descriptor content to the hardware DMA buffer descriptor. The main logic to control what's go into the buffer descriptor and which buffer descriptor to use is in the userspace AI engine library. Signed-off-by: Wendy Liang Reviewed-by: Hyun Kwon --- drivers/misc/xilinx-ai-engine/Makefile | 1 + drivers/misc/xilinx-ai-engine/ai-engine-aie.c | 19 + drivers/misc/xilinx-ai-engine/ai-engine-dma.c | 481 +++++++++++++++++++++ drivers/misc/xilinx-ai-engine/ai-engine-internal.h | 45 ++ drivers/misc/xilinx-ai-engine/ai-engine-part.c | 17 + include/uapi/linux/xlnx-ai-engine.h | 44 ++ 6 files changed, 607 insertions(+) create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-dma.c diff --git a/drivers/misc/xilinx-ai-engine/Makefile b/drivers/misc/xilinx-ai-engine/Makefile index 2dbed42..1b743fa 100644 --- a/drivers/misc/xilinx-ai-engine/Makefile +++ b/drivers/misc/xilinx-ai-engine/Makefile @@ -7,6 +7,7 @@ obj-$(CONFIG_XILINX_AIE) += xilinx-aie.o xilinx-aie-$(CONFIG_XILINX_AIE) := ai-engine-aie.o \ ai-engine-dev.o \ + ai-engine-dma.o \ ai-engine-mem.o \ ai-engine-part.o \ ai-engine-res.o \ diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-aie.c b/drivers/misc/xilinx-ai-engine/ai-engine-aie.c index 7fce2f00..ac95aff 100644 --- a/drivers/misc/xilinx-ai-engine/ai-engine-aie.c +++ b/drivers/misc/xilinx-ai-engine/ai-engine-aie.c @@ -107,6 +107,24 @@ static const struct aie_single_reg_field aie_col_clkbuf = { .regoff = AIE_SHIMPL_CLKCNTR_REGOFF, }; +static const struct aie_dma_attr aie_shimdma = { + .laddr = { + .mask = 0xffffffffU, + .regoff = 0U, + }, + .haddr = { + .mask = 0xffff0000U, + .regoff = 0x8U, + }, + .buflen = { + .mask = 0xffffffffU, + .regoff = 0x4U, + }, + .bd_regoff = 0x0001d000U, + .num_bds = 16, + .bd_len = 0x14U, +}; + static u32 aie_get_tile_type(struct aie_location *loc) { if (loc->row) @@ -232,6 +250,7 @@ int aie_device_init(struct aie_device *adev) adev->kernel_regs = aie_kernel_regs; adev->col_rst = &aie_col_rst; adev->col_clkbuf = &aie_col_clkbuf; + adev->shim_dma = &aie_shimdma; /* Get the columns resource */ /* Get number of columns from AI engine memory resource */ diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-dma.c b/drivers/misc/xilinx-ai-engine/ai-engine-dma.c new file mode 100644 index 0000000..863790b --- /dev/null +++ b/drivers/misc/xilinx-ai-engine/ai-engine-dma.c @@ -0,0 +1,481 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Xilinx AI Engine driver DMA implementation + * + * Copyright (C) 2020 Xilinx, Inc. + */ + +#include "ai-engine-internal.h" +#include +#include +#include +#include +#include +#include +#include + +/** + * struct aie_dmabuf - AI engine dmabuf information + * @attach: dmabuf attachment pointer + * @sgt: scatter/gather table + * @refs: refcount of the attached aie_dmabuf + * @node: list node + */ +struct aie_dmabuf { + struct dma_buf_attachment *attach; + struct sg_table *sgt; + refcount_t refs; + struct list_head node; +}; + +/** + * aie_part_find_dmabuf() - find a attached dmabuf + * @apart: AI engine partition + * @dmabuf: pointer to dmabuf + * @return: pointer to AI engine dmabuf struct of the found dmabuf, if dmabuf + * is not found, returns NULL. + * + * This function scans all the attached dmabufs to see the input dmabuf is + * in the list. if it is attached, return the corresponding struct aie_dmabuf + * pointer. + */ +static struct aie_dmabuf * +aie_part_find_dmabuf(struct aie_partition *apart, struct dma_buf *dmabuf) +{ + struct aie_dmabuf *adbuf; + + list_for_each_entry(adbuf, &apart->dbufs, node) { + if (dmabuf == adbuf->attach->dmabuf) + return adbuf; + } + + return NULL; +} + +/** + * aie_part_get_dmabuf_da_from_off() - get DMA address from offset to a dmabuf + * @apart: AI engine partition + * @dmabuf_fd: dmabuf file descriptor + * @off: offset to the start of a dmabuf + * @len: memory length + * @return: dma address, or 0 if @off or @len is invalid, or if @dmabuf_fd is + * not attached. + * + * This function returns DMA address if has been mapped to a dmabuf which has + * been attached to the AI engine partition. + */ +static dma_addr_t +aie_part_get_dmabuf_da_from_off(struct aie_partition *apart, int dmabuf_fd, + u64 off, size_t len) +{ + struct dma_buf *dbuf = dma_buf_get(dmabuf_fd); + struct aie_dmabuf *adbuf; + + if (IS_ERR(dbuf)) { + dev_err(&apart->dev, + "failed to get dma address, not able to get dmabuf from %d.\n", + dmabuf_fd); + return 0; + } + + adbuf = aie_part_find_dmabuf(apart, dbuf); + dma_buf_put(dbuf); + if (!adbuf) { + dev_err(&apart->dev, + "failed to get dma address, dmabuf %d not attached.\n", + dmabuf_fd); + return 0; + } + + if (off >= dbuf->size || off + len >= dbuf->size) { + dev_err(&apart->dev, + "failed to get dma address from buf %d, off=0x%llx, len=0x%zx.\n", + dmabuf_fd, off, len); + return 0; + } + + return sg_dma_address(adbuf->sgt->sgl) + off; +} + +/** + * aie_part_set_shimdma_bd() - Set the buffer descriptor to AI engine partition + * hardware + * @apart: AI engine partition + * @loc: AI engine tile location + * @bd_id: buffer descriptor ID + * @bd: pointer buffer descriptor content + * @return: 0 for success, negative value for failure + * + * This function sets the specified buffer descriptor content to the + * specified buffer descriptor in the specified AI engine SHIM NOC tile. + */ +static int aie_part_set_shimdma_bd(struct aie_partition *apart, + struct aie_location loc, u32 bd_id, u32 *bd) +{ + const struct aie_dma_attr *shim_dma = apart->adev->shim_dma; + struct aie_location loc_adjust; + u32 i, regoff, intile_regoff; + + intile_regoff = shim_dma->bd_regoff + shim_dma->bd_len * bd_id; + loc_adjust.col = loc.col + apart->range.start.col; + loc_adjust.row = loc.row + apart->range.start.row; + regoff = aie_cal_regoff(apart->adev, loc_adjust, intile_regoff); + + for (i = 0; i < shim_dma->bd_len / (sizeof(*bd)); + i++, regoff += sizeof(*bd)) + iowrite32(bd[i], apart->adev->base + regoff); + return 0; +} + +/** + * aie_part_validate_bdloc() - Validate SHIM DMA buffer descriptor location + * @apart: AI engine partition + * @loc: tile location + * @bd_id: buffer descriptor id + * + * @return: 0 for success, negative value for failure + * + * This function validate the SHIM DMA buffer descriptor base address. + */ +static int aie_part_validate_bdloc(struct aie_partition *apart, + struct aie_location loc, u32 bd_id) +{ + const struct aie_dma_attr *shim_dma = apart->adev->shim_dma; + struct aie_location loc_adjust; + u32 ttype; + + loc_adjust.col = loc.col + apart->range.start.col; + loc_adjust.row = loc.row + apart->range.start.row; + + if (aie_validate_location(apart, loc_adjust) < 0) { + dev_err(&apart->dev, + "invalid loc (%u,%u) in (%u,%u).\n", + loc.col, loc.row, + apart->range.size.col, apart->range.size.row); + return -EINVAL; + } + + ttype = apart->adev->ops->get_tile_type(&loc_adjust); + if (ttype != AIE_TILE_TYPE_SHIMNOC) { + dev_err(&apart->dev, + "failed to set bd, (%u,%u) is not SHIM NOC\n", + loc.col, loc.row); + return -EINVAL; + } + + if (bd_id >= shim_dma->num_bds) { + dev_err(&apart->dev, + "invalid SHIM DMA bd id: %u.\n", bd_id); + return -EINVAL; + } + + return 0; +} + +/** + * aie_part_attach_dmabuf() - Attach dmabuf to an AI engine + * @apart: AI engine partition + * @dbuf: pointer to the DMA buffer to attach + * @return: pointer to AI engine dmabuf structure for success, or error value + * for failure + * + * This function attaches a dmabuf to the specified AI engine partition. + */ +static struct aie_dmabuf *aie_part_attach_dmabuf(struct aie_partition *apart, + struct dma_buf *dbuf) +{ + struct aie_dmabuf *adbuf; + struct dma_buf_attachment *attach; + struct sg_table *sgt; + + attach = dma_buf_attach(dbuf, &apart->dev); + if (IS_ERR(attach)) { + dev_err(&apart->dev, "failed to attach dmabuf\n"); + return ERR_CAST(attach); + } + + sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL); + if (IS_ERR(sgt)) { + dev_err(&apart->dev, "failed to map dmabuf attachment\n"); + dma_buf_detach(dbuf, attach); + return ERR_CAST(sgt); + } + + if (sgt->nents != 1) { + dma_addr_t next_sg_addr = sg_dma_address(sgt->sgl); + struct scatterlist *s; + unsigned int i; + + for_each_sg(sgt->sgl, s, sgt->nents, i) { + if (sg_dma_address(s) != next_sg_addr) { + dev_err(&apart->dev, + "dmabuf not contiguous\n"); + dma_buf_unmap_attachment(attach, sgt, + attach->dir); + dma_buf_detach(dbuf, attach); + return ERR_PTR(-EINVAL); + } + + next_sg_addr = sg_dma_address(s) + sg_dma_len(s); + } + } + + adbuf = devm_kzalloc(&apart->dev, sizeof(*adbuf), GFP_KERNEL); + if (!adbuf) { + dma_buf_unmap_attachment(attach, sgt, attach->dir); + dma_buf_detach(dbuf, attach); + return ERR_PTR(-ENOMEM); + } + + adbuf->attach = attach; + /* + * dmabuf attachment doesn't always include the sgt, store it in + * AI engine dma buf structure. + */ + adbuf->sgt = sgt; + + refcount_set(&adbuf->refs, 1); + + list_add(&adbuf->node, &apart->dbufs); + return adbuf; +} + +/** + * aie_part_dmabuf_attach_get() - Get reference to an dmabuf attachment + * @adbuf: AI engine partition attached dmabuf + * + * This call will increase the reference count by 1 + */ +static void aie_part_dmabuf_attach_get(struct aie_dmabuf *adbuf) +{ + refcount_inc(&adbuf->refs); +} + +/** + * aie_part_dmabuf_attach_put() - Put reference to an dmabuf attachment + * @adbuf: AI engine partition attached dmabuf + * + * This call will decrease the reference count by 1. If the refcount reaches + * 0, it will detach the dmabuf. + */ +static void aie_part_dmabuf_attach_put(struct aie_dmabuf *adbuf) +{ + struct dma_buf *dbuf; + + if (!refcount_dec_and_test(&adbuf->refs)) + return; + + dbuf = adbuf->attach->dmabuf; + dma_buf_unmap_attachment(adbuf->attach, adbuf->sgt, adbuf->attach->dir); + dma_buf_detach(dbuf, adbuf->attach); + dma_buf_put(dbuf); + list_del(&adbuf->node); +} + +/** + * aie_part_release_dmabufs() - detach all the attached dmabufs from partition + * @apart: AI engine partition + */ +void aie_part_release_dmabufs(struct aie_partition *apart) +{ + struct aie_dmabuf *adbuf, *tmpadbuf; + + list_for_each_entry_safe(adbuf, tmpadbuf, &apart->dbufs, node) { + struct dma_buf *dbuf = adbuf->attach->dmabuf; + + dma_buf_unmap_attachment(adbuf->attach, adbuf->sgt, + adbuf->attach->dir); + dma_buf_detach(dbuf, adbuf->attach); + dma_buf_put(dbuf); + list_del(&adbuf->node); + devm_kfree(&apart->dev, adbuf); + } +} + +/** + * aie_part_attach_dmabuf_req() - Handle attaching dmabuf to an AI engine + * partition request + * @apart: AI engine partition + * @user_args: user AI engine dmabuf argument + * + * @return: 0 for success, negative value for failure + * + * This function attaches a dmabuf to the specified AI engine partition and map + * the attachment. It checks if the dmabuf is already attached, if it is not + * attached, attach it. It returns the number of entries of the attachment to + * the AI engine dmabuf user argument. If user wants to know the sg list, it + * can use AI engine get sg ioctl. + */ +long aie_part_attach_dmabuf_req(struct aie_partition *apart, + void __user *user_args) +{ + struct aie_dmabuf *adbuf; + struct dma_buf *dbuf; + long ret; + int dmabuf_fd = (int)(uintptr_t)user_args; + + dbuf = dma_buf_get(dmabuf_fd); + if (IS_ERR(dbuf)) { + dev_err(&apart->dev, "failed to get dmabuf from %d.\n", + dmabuf_fd); + return PTR_ERR(dbuf); + } + + ret = mutex_lock_interruptible(&apart->mlock); + if (ret) { + dma_buf_put(dbuf); + return ret; + } + + adbuf = aie_part_find_dmabuf(apart, dbuf); + if (!adbuf) + adbuf = aie_part_attach_dmabuf(apart, dbuf); + else + aie_part_dmabuf_attach_get(adbuf); + + mutex_unlock(&apart->mlock); + + if (IS_ERR(adbuf)) { + dev_err(&apart->dev, "failed to attach dmabuf\n"); + dma_buf_put(dbuf); + return PTR_ERR(adbuf); + } + + return 0; +} + +/** + * aie_part_detach_dmabuf_req() - Handle detaching dmabuf from an AI engine + * partition request + * @apart: AI engine partition + * @user_args: user AI engine dmabuf argument + * + * @return: 0 for success, negative value for failure + * + * This function unmaps and detaches a dmabuf from the specified AI engine + * partition. + */ +long aie_part_detach_dmabuf_req(struct aie_partition *apart, + void __user *user_args) +{ + int dmabuf_fd; + struct dma_buf *dbuf; + struct aie_dmabuf *adbuf; + int ret; + + dmabuf_fd = (int)(uintptr_t)user_args; + + dbuf = dma_buf_get(dmabuf_fd); + if (IS_ERR(dbuf)) { + dev_err(&apart->dev, "failed to get dmabuf %d.\n", dmabuf_fd); + return PTR_ERR(dbuf); + } + + ret = mutex_lock_interruptible(&apart->mlock); + if (ret) { + dma_buf_put(dbuf); + return ret; + } + + adbuf = aie_part_find_dmabuf(apart, dbuf); + dma_buf_put(dbuf); + if (!adbuf) { + dev_err(&apart->dev, "failed to find dmabuf %d.\n", dmabuf_fd); + mutex_unlock(&apart->mlock); + return -EINVAL; + } + + aie_part_dmabuf_attach_put(adbuf); + + mutex_unlock(&apart->mlock); + + return 0; +} + +/** + * aie_part_set_dmabuf_bd() - Set AI engine SHIM DMA dmabuf buffer descriptor + * @apart: AI engine partition + * @user_args: user AI engine dmabuf argument + * + * @return: 0 for success, negative value for failure + * + * This function set the user specified buffer descriptor into the SHIM DMA + * buffer descriptor. The buffer descriptor contained in the @user_args has the + * offset to the start of the buffer descriptor. + */ +long aie_part_set_dmabuf_bd(struct aie_partition *apart, + void __user *user_args) +{ + struct aie_device *adev = apart->adev; + const struct aie_dma_attr *shim_dma = adev->shim_dma; + struct aie_dmabuf_bd_args args; + u32 *bd, *tmpbd, len, laddr, haddr, regval; + u64 off; + dma_addr_t addr; + int ret; + + if (copy_from_user(&args, user_args, sizeof(args))) + return -EFAULT; + + ret = aie_part_validate_bdloc(apart, args.loc, args.bd_id); + if (ret) { + dev_err(&apart->dev, "invalid SHIM DMA BD reg address.\n"); + return -EINVAL; + } + + bd = memdup_user(u64_to_user_ptr(args.bd), shim_dma->bd_len); + if (IS_ERR(bd)) + return PTR_ERR(bd); + + regval = bd[shim_dma->buflen.regoff / sizeof(u32)]; + len = aie_get_reg_field(&shim_dma->buflen, regval); + if (!len) { + dev_err(&apart->dev, "no buf length from shim dma bd.\n"); + kfree(bd); + return -EINVAL; + } + + /* Get low 32bit address offset */ + tmpbd = (u32 *)((char *)bd + shim_dma->laddr.regoff); + laddr = *tmpbd & shim_dma->laddr.mask; + /* Get high 32bit address offset */ + tmpbd = (u32 *)((char *)bd + shim_dma->haddr.regoff); + haddr = *tmpbd & shim_dma->haddr.mask; + off = laddr | ((u64)haddr << 32); + + ret = mutex_lock_interruptible(&apart->mlock); + if (ret) { + kfree(bd); + return ret; + } + + /* Get device address from offset */ + addr = aie_part_get_dmabuf_da_from_off(apart, args.buf_fd, off, len); + if (!addr) { + dev_err(&apart->dev, "invalid buffer 0x%llx, 0x%x.\n", + off, len); + mutex_unlock(&apart->mlock); + kfree(bd); + return -EINVAL; + } + + /* Set low 32bit address */ + laddr = lower_32_bits(addr); + tmpbd = (u32 *)((char *)bd + shim_dma->laddr.regoff); + *tmpbd &= ~shim_dma->laddr.mask; + *tmpbd |= aie_get_field_val(&shim_dma->laddr, laddr); + + /* Set high 32bit address */ + haddr = upper_32_bits(addr); + tmpbd = (u32 *)((char *)bd + shim_dma->haddr.regoff); + *tmpbd &= ~shim_dma->haddr.mask; + *tmpbd |= aie_get_field_val(&shim_dma->haddr, haddr); + + ret = aie_part_set_shimdma_bd(apart, args.loc, args.bd_id, bd); + mutex_unlock(&apart->mlock); + if (ret) + dev_err(&apart->dev, "failed to set to shim dma bd.\n"); + + kfree(bd); + return ret; +} diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-internal.h b/drivers/misc/xilinx-ai-engine/ai-engine-internal.h index e84610b..bf3a09c 100644 --- a/drivers/misc/xilinx-ai-engine/ai-engine-internal.h +++ b/drivers/misc/xilinx-ai-engine/ai-engine-internal.h @@ -90,6 +90,24 @@ struct aie_part_mem { }; /** + * struct aie_dma_attr - AI engine DMA attributes structure + * @laddr: low address field attributes + * @haddr: high address field attributes + * @buflen: buffer length field attributes + * @bd_regoff: SHIM DMA buffer descriptors register offset + * @num_bds: number of buffer descriptors + * @bd_len: length of a buffer descriptor in bytes + */ +struct aie_dma_attr { + struct aie_single_reg_field laddr; + struct aie_single_reg_field haddr; + struct aie_single_reg_field buflen; + u32 bd_regoff; + u32 num_bds; + u32 bd_len; +}; + +/** * struct aie_tile_operations - AI engine device operations * @get_tile_type: get type of tile based on tile operation * @get_mem_info: get different types of memories information @@ -127,6 +145,7 @@ struct aie_resource { * @ops: tile operations * @col_rst: column reset attribute * @col_clkbuf: column clock buffer attribute + * @shim_dma: SHIM DMA attribute * @size: size of the AI engine address space * @array_shift: array address shift * @col_shift: column address shift @@ -147,6 +166,7 @@ struct aie_device { const struct aie_tile_operations *ops; const struct aie_single_reg_field *col_rst; const struct aie_single_reg_field *col_clkbuf; + const struct aie_dma_attr *shim_dma; size_t size; struct aie_resource cols_res; u32 array_shift; @@ -159,6 +179,7 @@ struct aie_device { /** * struct aie_partition - AI engine partition structure * @node: list node + * @dbufs: dmabufs list * @adev: pointer to AI device instance * @pmems: pointer to partition memories types * @range: range of partition @@ -172,6 +193,7 @@ struct aie_device { */ struct aie_partition { struct list_head node; + struct list_head dbufs; struct aie_device *adev; struct aie_part_mem *pmems; struct aie_range range; @@ -229,6 +251,20 @@ static inline u32 aie_get_field_val(const struct aie_single_reg_field *field, } /** + * aie_get_reg_field() - get value from a field from a register valuer + * @field: a field in a register + * @regval: register value + * @return: value of a register field + */ +static inline u32 aie_get_reg_field(const struct aie_single_reg_field *field, + u32 regval) +{ + long long mask64 = (long long)field->mask & 0x00000000ffffffff; + + return (regval & field->mask) >> __bf_shf(mask64); +} + +/** * aie_cal_regoff() - calculate register offset to the whole AI engine * device start address * @adev: AI engine device @@ -286,5 +322,14 @@ int aie_part_clean(struct aie_partition *apart); int aie_mem_get_info(struct aie_partition *apart, unsigned long arg); +long aie_part_attach_dmabuf_req(struct aie_partition *apart, + void __user *user_args); +long aie_part_detach_dmabuf_req(struct aie_partition *apart, + void __user *user_args); +long aie_part_set_bd(struct aie_partition *apart, void __user *user_args); +long aie_part_set_dmabuf_bd(struct aie_partition *apart, + void __user *user_args); +void aie_part_release_dmabufs(struct aie_partition *apart); + int aie_device_init(struct aie_device *adev); #endif /* AIE_INTERNAL_H */ diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-part.c b/drivers/misc/xilinx-ai-engine/ai-engine-part.c index 4be6d38..dcfb9ec 100644 --- a/drivers/misc/xilinx-ai-engine/ai-engine-part.c +++ b/drivers/misc/xilinx-ai-engine/ai-engine-part.c @@ -8,6 +8,7 @@ #include #include #include +#include #include #include #include @@ -221,6 +222,7 @@ static int aie_part_release(struct inode *inode, struct file *filp) if (ret) return ret; + aie_part_release_dmabufs(apart); aie_part_clean(apart); apart->status = 0; @@ -296,6 +298,12 @@ static long aie_part_ioctl(struct file *fp, unsigned int cmd, unsigned long arg) } case AIE_GET_MEM_IOCTL: return aie_mem_get_info(apart, arg); + case AIE_ATTACH_DMABUF_IOCTL: + return aie_part_attach_dmabuf_req(apart, argp); + case AIE_DETACH_DMABUF_IOCTL: + return aie_part_detach_dmabuf_req(apart, argp); + case AIE_SET_SHIMDMA_DMABUF_BD_IOCTL: + return aie_part_set_dmabuf_bd(apart, argp); default: dev_err(&apart->dev, "Invalid ioctl command %u.\n", cmd); ret = -EINVAL; @@ -422,6 +430,7 @@ static struct aie_partition *aie_create_partition(struct aie_device *adev, return ERR_PTR(-ENOMEM); apart->adev = adev; + INIT_LIST_HEAD(&apart->dbufs); memcpy(&apart->range, range, sizeof(*range)); mutex_init(&apart->mlock); @@ -443,6 +452,10 @@ static struct aie_partition *aie_create_partition(struct aie_device *adev, return ERR_PTR(ret); } + /* Set up the DMA mask */ + dev->coherent_dma_mask = DMA_BIT_MASK(48); + dev->dma_mask = &dev->coherent_dma_mask; + /* * Create array to keep the information of the different types of tile * memories information of the AI engine partition. @@ -521,6 +534,10 @@ of_aie_part_probe(struct aie_device *adev, struct device_node *nc) apart->dev.of_node = nc; apart->partition_id = partition_id; + ret = of_dma_configure(&apart->dev, nc, true); + if (ret) + dev_warn(&apart->dev, "Failed to configure DMA.\n"); + dev_info(&adev->dev, "AI engine part(%u,%u),(%u,%u), id %u is probed successfully.\n", range.start.col, range.start.row, diff --git a/include/uapi/linux/xlnx-ai-engine.h b/include/uapi/linux/xlnx-ai-engine.h index 9faeebe..75d9dbf 100644 --- a/include/uapi/linux/xlnx-ai-engine.h +++ b/include/uapi/linux/xlnx-ai-engine.h @@ -130,6 +130,22 @@ struct aie_partition_req { __u32 flag; }; +/** + * struct aie_dmabuf_bd_args - AIE dmabuf buffer descriptor information + * @bd: DMA buffer descriptor, within the buffer descriptor, the address field + * will be the offset to the start of the dmabuf. It is an array of __u32 + * words. + * @buf_fd: DMA buffer handler which is dmabuf file descriptor + * @loc: Tile location relative to the start of a partition + * @bd_id: buffer descriptor id + */ +struct aie_dmabuf_bd_args { + __u64 bd; + struct aie_location loc; + int buf_fd; + __u32 bd_id; +}; + #define AIE_IOCTL_BASE 'A' /* AI engine device IOCTL operations */ @@ -160,4 +176,32 @@ struct aie_partition_req { */ #define AIE_GET_MEM_IOCTL _IOWR(AIE_IOCTL_BASE, 0x9, \ struct aie_mem_args) +/** + * DOC: AIE_ATTACH_DMABUF_IOCTL - attach a dmabuf to AI engine partition + * + * This ioctl is used to attach a dmabuf to the AI engine partition. AI engine + * partition will return the number of scatter gather list elements of the + * dmabuf. + */ +#define AIE_ATTACH_DMABUF_IOCTL _IOR(AIE_IOCTL_BASE, 0xa, int) + +/** + * DOC: AIE_DETACH_DMABUF_IOCTL - dettach a dmabuf from AI engine partition + * + * This ioctl is used to detach a dmabuf from the AI engine partition + */ +#define AIE_DETACH_DMABUF_IOCTL _IOR(AIE_IOCTL_BASE, 0xb, int) + +/** + * DOC: AIE_SET_SHIMDMA_DMABUF_BD_IOCTL - set buffer descriptor which contains + * dmabuf to SHIM DMA + * + * This ioctl is used to set the buffer descriptor to SHIM DMA. The + * aie_dmabuf_bd_args contains the dmabuf fd and the buffer descriptor contents. + * The address field in the buffer descriptor contents should be the offset to + * the start of the dmabuf. + */ +#define AIE_SET_SHIMDMA_DMABUF_BD_IOCTL _IOW(AIE_IOCTL_BASE, 0x10, \ + struct aie_dmabuf_bd_args) + #endif From patchwork Mon Nov 30 07:48:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaying Liang X-Patchwork-Id: 334947 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBC93C8301F for ; Mon, 30 Nov 2020 07:50:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6C8C620857 for ; Mon, 30 Nov 2020 07:50:13 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=xilinx.onmicrosoft.com header.i=@xilinx.onmicrosoft.com header.b="OjrFfLb1" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727535AbgK3Htj (ORCPT ); Mon, 30 Nov 2020 02:49:39 -0500 Received: from mail-dm6nam11on2055.outbound.protection.outlook.com ([40.107.223.55]:25216 "EHLO NAM11-DM6-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727434AbgK3Htj (ORCPT ); Mon, 30 Nov 2020 02:49:39 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mgxaLxHdztCg8i/uo/RBNiionGW4Rgqi1F9PG8owXWh/kxpMca8IjHAapai6PpHwz0X6Qc1tsALupOCeLdSEXeKQNQtIef/n2L2aqsDUAP6f62iAVsTJ18qxtn7s54TTgG2Cp8YIEKYYa8divWOR8Q5cWP8Ch7ELSdgSlWL7dwSCnUMQT1vdBWMvB65aJeEy24Snq5gaf5V5OYMIrfHaYqV2UgBUM4LCUCHEA3plUFfkj1A7HLNCpZIFOQo82uwycP36cQOP8hMH/6NioTXlULqnUyPVvweC7ArP2BxtaONSVmMgTYO8c+KJotwYaUnfR8xm2uDJRstDwnpBKulaPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vghQNYKIxc54dw5ls7dzSN5SAYnmfHZqMdclWTaJaoo=; b=VxfeUGud1gpQ7PGNdtjEQLY/oNpv1bJ5u7nw/RzUx13WqwLWwKaYqyvsJJt7N0PdEYLH2YJI0IpZWrrOGh4DcIB53F7cIdH5WilzJu7hnoj4N7kS1p3tg3WURFPsi9TTJw3dPloJ1BjJdq/FRUCafek62weQoTD7RbOnsF2YWZvH4Hg22rmut4eeYSQpPYS2jV9GIDBSeoRhXhWda/6QOfh+gvzpDmhxZmofIOefleqDX2Df11KoeRN93/Fw9Drxrmv1tneMBLpXsl0rvteBBhGIXqR2NCgOkVuW8jxEs7mN2H8cgLfhTVfkbJ7Dg7U12SGU+vL/9OTyq03QEppWKQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.62.198) smtp.rcpttodomain=kernel.org smtp.mailfrom=xilinx.com; dmarc=bestguesspass action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vghQNYKIxc54dw5ls7dzSN5SAYnmfHZqMdclWTaJaoo=; b=OjrFfLb1UQU98DNk9tFLsDiN1pPVFGRSLlIegXsNkakBnu48Q8wM30bWo+2tlKCSrq39ajiUbcz3yhCfjF8OxCmHyfxLV0gOxJcwfjM6fKEk8N7PmfQY2J/NB27XhN7sIblykFdWes7MLSC17xwRIKEX9wLVcI2IyBrfB3giDE8= Received: from CY4PR03CA0023.namprd03.prod.outlook.com (2603:10b6:903:33::33) by BN7PR02MB3971.namprd02.prod.outlook.com (2603:10b6:406:fe::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3611.23; Mon, 30 Nov 2020 07:48:45 +0000 Received: from CY1NAM02FT037.eop-nam02.prod.protection.outlook.com (2603:10b6:903:33:cafe::1d) by CY4PR03CA0023.outlook.office365.com (2603:10b6:903:33::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3611.20 via Frontend Transport; Mon, 30 Nov 2020 07:48:45 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198) smtp.mailfrom=xilinx.com; kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=bestguesspass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.62.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.62.198; helo=xsj-pvapexch02.xlnx.xilinx.com; Received: from xsj-pvapexch02.xlnx.xilinx.com (149.199.62.198) by CY1NAM02FT037.mail.protection.outlook.com (10.152.75.77) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.3611.27 via Frontend Transport; Mon, 30 Nov 2020 07:48:45 +0000 Received: from xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1913.5; Sun, 29 Nov 2020 23:48:31 -0800 Received: from smtp.xilinx.com (172.19.127.96) by xsj-pvapexch02.xlnx.xilinx.com (172.19.86.41) with Microsoft SMTP Server id 15.1.1913.5 via Frontend Transport; Sun, 29 Nov 2020 23:48:31 -0800 Envelope-to: michal.simek@xilinx.com, derek.kiernan@xilinx.com, dragan.cvetic@xilinx.com, rajan.vaja@xilinx.com, tejas.patel@xilinx.com, manish.narani@xilinx.com, ravi.patel@xilinx.com, izhar.ameer.shaikh@xilinx.com, wendy.liang@xilinx.com, robh+dt@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, sumit.semwal@linaro.org, christian.koenig@amd.com, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org Received: from [172.19.2.167] (port=48528 helo=xsjjliang50.xilinx.com) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1kjdvD-0006Pm-4f; Sun, 29 Nov 2020 23:48:31 -0800 From: Wendy Liang To: , , , , , , , , , , , CC: , , , , , Izhar Ameer Shaikh , Wendy Liang Subject: [PATCH v3 8/9] firmware: xilinx: Add IOCTL support for AIE ISR Clear Date: Sun, 29 Nov 2020 23:48:24 -0800 Message-ID: <1606722505-16194-9-git-send-email-wendy.liang@xilinx.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1606722505-16194-1-git-send-email-wendy.liang@xilinx.com> References: <1606722505-16194-1-git-send-email-wendy.liang@xilinx.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 82d49edd-55f1-4d09-9659-08d895045d61 X-MS-TrafficTypeDiagnostic: BN7PR02MB3971: X-Microsoft-Antispam-PRVS: X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply X-MS-Oob-TLC-OOBClassifiers: OLM:7219; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qf7hzy1PSFgS8g03MpFgbA7Sbgt3ZzIZbVZYEXO3uXvoC/BxuXfm7QkLNLf9EjJ6qpgz2RXkAxlTDt4Sf/7JBHu4dVfD2JuHxlMzeUdXFeth4flGAakaHV3/8zYt1USuKW9Iw/yQGD5vb7zWlrKWBV9e1gvKMnIaN+hFkay5ASCKfXT3afXJp5beLeeBZzeE7YCEjfAuAKRd2SrEiycxOk8rBwhcX7S9SJthmT3OsQcdMCo1X8eAcSveWqIUgI7uSj2T/gtOvfjthpoL7l9gS3ncAAGZRobeSZbe3ZTl1w4YTKyuzucbIWa6bbFgvsjJJEx1dLMBqqCZE/21cRipqCExnGRMZuRvPsrP65vFCwls7OYuFxzfB1XzNoTAT0eLTfXU8l4mxReg/qXMt7Jwqv0M1N9X8NTZrrvoFCxmqTeJuLJ8nGxSPc+yP8oquD7a X-Forefront-Antispam-Report: CIP:149.199.62.198; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:xsj-pvapexch02.xlnx.xilinx.com; PTR:unknown-62-198.xilinx.com; CAT:NONE; SFS:(4636009)(39850400004)(346002)(136003)(396003)(376002)(46966005)(107886003)(36756003)(6666004)(7696005)(47076004)(2616005)(70206006)(70586007)(356005)(83380400001)(9786002)(7636003)(336012)(82310400003)(6636002)(426003)(5660300002)(8676002)(54906003)(2906002)(8936002)(26005)(478600001)(186003)(82740400003)(36906005)(110136005)(4326008)(921005)(316002)(44832011)(7416002)(102446001); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Nov 2020 07:48:45.1349 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 82d49edd-55f1-4d09-9659-08d895045d61 X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.62.198]; Helo=[xsj-pvapexch02.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: CY1NAM02FT037.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR02MB3971 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Izhar Ameer Shaikh Latching of AIE NPI Interrupts is present in Versal ES1 Silicon Rev, however it has been removed from ES2 rev. As a result on ES1, in order to use the interrupt, a client needs to request PMC to clear/ack the interrupt. Provide an EEMI IOCTL to serve the same purpose. Note that, this will only be applicable for ES1 rev. For ES2 and other non-silicon platforms, this call will essentially be a NOP in the firmware. Signed-off-by: Izhar Ameer Shaikh Signed-off-by: Wendy Liang --- drivers/firmware/xilinx/zynqmp.c | 14 ++++++++++++++ include/linux/firmware/xlnx-zynqmp.h | 8 ++++++++ 2 files changed, 22 insertions(+) diff --git a/drivers/firmware/xilinx/zynqmp.c b/drivers/firmware/xilinx/zynqmp.c index d08ac82..23e58cc 100644 --- a/drivers/firmware/xilinx/zynqmp.c +++ b/drivers/firmware/xilinx/zynqmp.c @@ -729,6 +729,20 @@ int zynqmp_pm_set_boot_health_status(u32 value) } /** + * zynqmp_pm_clear_aie_npi_isr - Clear AI engine NPI interrupt status register + * @node: AI engine node id + * @irq_mask: Mask of AI engine NPI interrupt bit to clear + * + * Return: Returns status, either success or error+reason + */ +int zynqmp_pm_clear_aie_npi_isr(u32 node, u32 irq_mask) +{ + return zynqmp_pm_invoke_fn(PM_IOCTL, node, IOCTL_AIE_ISR_CLEAR, + irq_mask, 0, NULL); +} +EXPORT_SYMBOL_GPL(zynqmp_pm_clear_aie_npi_isr); + +/** * zynqmp_pm_reset_assert - Request setting of reset (1 - assert, 0 - release) * @reset: Reset to be configured * @assert_flag: Flag stating should reset be asserted (1) or diff --git a/include/linux/firmware/xlnx-zynqmp.h b/include/linux/firmware/xlnx-zynqmp.h index 83ac9ec..defa4ea 100644 --- a/include/linux/firmware/xlnx-zynqmp.h +++ b/include/linux/firmware/xlnx-zynqmp.h @@ -114,6 +114,8 @@ enum pm_ioctl_id { IOCTL_READ_PGGS = 15, /* Set healthy bit value */ IOCTL_SET_BOOT_HEALTH_STATUS = 17, + /* AI engine NPI ISR clear */ + IOCTL_AIE_ISR_CLEAR = 24, }; enum pm_query_id { @@ -355,6 +357,7 @@ int zynqmp_pm_write_pggs(u32 index, u32 value); int zynqmp_pm_read_pggs(u32 index, u32 *value); int zynqmp_pm_system_shutdown(const u32 type, const u32 subtype); int zynqmp_pm_set_boot_health_status(u32 value); +int zynqmp_pm_clear_aie_npi_isr(u32 node, u32 irq_mask); #else static inline struct zynqmp_eemi_ops *zynqmp_pm_get_eemi_ops(void) { @@ -505,6 +508,11 @@ static inline int zynqmp_pm_set_boot_health_status(u32 value) { return -ENODEV; } + +static inline int zynqmp_pm_clear_aie_npi_isr(u32 node, u32 irq_mask) +{ + return -ENODEV; +} #endif #endif /* __FIRMWARE_ZYNQMP_H__ */ From patchwork Mon Nov 30 07:48:25 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jiaying Liang X-Patchwork-Id: 334946 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 557A9C8301C for ; Mon, 30 Nov 2020 07:50:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0597E20719 for ; Mon, 30 Nov 2020 07:50:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=xilinx.onmicrosoft.com header.i=@xilinx.onmicrosoft.com header.b="IBtRHaw0" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727770AbgK3Hts (ORCPT ); Mon, 30 Nov 2020 02:49:48 -0500 Received: from mail-bn8nam12on2078.outbound.protection.outlook.com ([40.107.237.78]:54272 "EHLO NAM12-BN8-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727519AbgK3Htr (ORCPT ); Mon, 30 Nov 2020 02:49:47 -0500 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=G60OTeKqCXMHO6T0+ceXCukzz/xKYvOacMM/IDn5niAs7k639ipXQXSF0xHTi85mXJCwOpCRvmLOMUmvfRiSmaMnjmDJduxlUNTc9FXdtD8aujh5zyusivkYxTQE8HOluzBfxPo0zQ2gg2h28vBQ33Y926adA0oBDS1YKdP1cA0IDCn97qPhFfpKylb5QEGqydxILB17MW2a5/2wMeAKfiGNp+N3yAh+1/cj8pIMbo42nMb3v1XjyWMUZwD2u56x8KKAEh+00ru+3KDIBFPIedNciuMtYsk8pqQQpwhxxZobqIKligBSYZinhO/SW5yA88xfzuHyRj9jvmP1F3F53Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=v+cW3gEpjQ8qn2AClHZpnzgw+EM27UiEVGGWREUIKzA=; b=U3NqmJeQamlaFdxtCbIPGWcpYJLp2NqsS6geFloA8/wi9vd4fHEmrv7CBcfVeGXZh5l0mTCbSH69ljIY6OeXLV9K3uoDQoZ7WiOajsVYY8Vj2vkeeYXwbP37gn5Dr11fl+0rCIphYUtYcR2J3gIEZHG3v6DOhXWUZdKIWqnGY3m223a6cEGl8YbwXlgAfDwG+9WrnaPcZcFrWcBNgImpGp0uEz9H+QxPCJ+D4xY/gwycA+MHr+iLbhsXTT1N+zftjdN2Yh9VxCLMDonTsYqAuC6uKw22Pd7Kc9vpZkiRRXrlmqtO1A+zQmm4lzK8nJFtybGXbkzKeTFybkWLHm07QQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 149.199.62.198) smtp.rcpttodomain=kernel.org smtp.mailfrom=xilinx.com; dmarc=bestguesspass action=none header.from=xilinx.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xilinx.onmicrosoft.com; s=selector2-xilinx-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=v+cW3gEpjQ8qn2AClHZpnzgw+EM27UiEVGGWREUIKzA=; b=IBtRHaw0WXOyhEqxZBhD/Pg1dvtDZXO5GRLb69wYPAknn/bqQFiVZc0yQUWZ3/ytk7Db6QwkDcTek4ZJ8jz0KrGWFUETcKPP3NsSZM6ox8AlaXCrbL6+qoBPQWkzO1FSvBcSRCrIIHwEWh0EHk1f3qcnegu6JKRTH8PSE6UI45g= Received: from SA9PR10CA0010.namprd10.prod.outlook.com (2603:10b6:806:a7::15) by BN7PR02MB5041.namprd02.prod.outlook.com (2603:10b6:408:2e::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3611.24; Mon, 30 Nov 2020 07:48:46 +0000 Received: from SN1NAM02FT051.eop-nam02.prod.protection.outlook.com (2603:10b6:806:a7:cafe::a7) by SA9PR10CA0010.outlook.office365.com (2603:10b6:806:a7::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3611.25 via Frontend Transport; Mon, 30 Nov 2020 07:48:46 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 149.199.62.198) smtp.mailfrom=xilinx.com; kernel.org; dkim=none (message not signed) header.d=none;kernel.org; dmarc=bestguesspass action=none header.from=xilinx.com; Received-SPF: Pass (protection.outlook.com: domain of xilinx.com designates 149.199.62.198 as permitted sender) receiver=protection.outlook.com; client-ip=149.199.62.198; helo=xsj-pvapexch01.xlnx.xilinx.com; Received: from xsj-pvapexch01.xlnx.xilinx.com (149.199.62.198) by SN1NAM02FT051.mail.protection.outlook.com (10.152.73.103) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.3611.27 via Frontend Transport; Mon, 30 Nov 2020 07:48:46 +0000 Received: from xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) by xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1913.5; Sun, 29 Nov 2020 23:48:31 -0800 Received: from smtp.xilinx.com (172.19.127.96) by xsj-pvapexch01.xlnx.xilinx.com (172.19.86.40) with Microsoft SMTP Server id 15.1.1913.5 via Frontend Transport; Sun, 29 Nov 2020 23:48:31 -0800 Envelope-to: michal.simek@xilinx.com, derek.kiernan@xilinx.com, dragan.cvetic@xilinx.com, rajan.vaja@xilinx.com, tejas.patel@xilinx.com, manish.narani@xilinx.com, ravi.patel@xilinx.com, nishad.saraf@xilinx.com, wendy.liang@xilinx.com, robh+dt@kernel.org, arnd@arndb.de, gregkh@linuxfoundation.org, sumit.semwal@linaro.org, christian.koenig@amd.com, devicetree@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org Received: from [172.19.2.167] (port=48528 helo=xsjjliang50.xilinx.com) by smtp.xilinx.com with esmtp (Exim 4.90) (envelope-from ) id 1kjdvD-0006Pm-5c; Sun, 29 Nov 2020 23:48:31 -0800 From: Wendy Liang To: , , , , , , , , , , , CC: , , , , , Nishad Saraf , Wendy Liang Subject: [PATCH v3 9/9] misc: xilinx-ai-engine: Add support for servicing error interrupts Date: Sun, 29 Nov 2020 23:48:25 -0800 Message-ID: <1606722505-16194-10-git-send-email-wendy.liang@xilinx.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1606722505-16194-1-git-send-email-wendy.liang@xilinx.com> References: <1606722505-16194-1-git-send-email-wendy.liang@xilinx.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 190ee4eb-4b9a-4f74-b159-08d895045e0d X-MS-TrafficTypeDiagnostic: BN7PR02MB5041: X-Microsoft-Antispam-PRVS: X-Auto-Response-Suppress: DR, RN, NRN, OOF, AutoReply X-MS-Oob-TLC-OOBClassifiers: OLM:3173; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 7Hdpv2/ExqwTdu8SybSR869boLYBFLTPK3NObNLilKDNhLBwQkQynMpLit8yoYVkRs/YR6lFYoCO64A5kY2W82KTANrhS3mZcewcrYYQRNasrMKUQ+T6w9F1OY7Arnd3cb3+uCBzWumNbhOwMFAWVDn/SJv4f7Zvo4rPoU3+34uzQW4d/963NtwVg7RjmP8UUcHkkwOLInQTtPQe1SNDtHE0rejUXTvO1yEBHk1OC3zYy5FzEJgo1/tnDo1E8aPB1M7DS9lY7q996kprTVWWtFeDJR84P/BDCG1i+R5KE+6aXR1KgBUaREgMuliSsI6Nc62hQJGAgPVgsdtrHSv/7Zp/YXSkWzAFQ1tsPt0/BLnAQ59zcLjt3zi99PJKtmVbC4bmncxn3B7hubR+NlUOuIv7KdqwaxbA1TMSft49xwaIukq2qQGubV/FiVlsSYuaT932D/Z0gxcvq8zFhJ3OCjsZlxB1G7JgALsysx+cv8pYppQMxfzEEeCaEoAzmAwW X-Forefront-Antispam-Report: CIP:149.199.62.198; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:xsj-pvapexch01.xlnx.xilinx.com; PTR:unknown-62-198.xilinx.com; CAT:NONE; SFS:(4636009)(376002)(136003)(346002)(39850400004)(396003)(46966005)(6666004)(26005)(54906003)(82310400003)(7416002)(8676002)(2906002)(83380400001)(4326008)(107886003)(6636002)(44832011)(921005)(7696005)(30864003)(36756003)(9786002)(478600001)(8936002)(36906005)(316002)(2616005)(5660300002)(186003)(336012)(70206006)(70586007)(47076004)(7636003)(356005)(426003)(110136005)(82740400003)(102446001)(579004)(473944003)(414714003); DIR:OUT; SFP:1101; X-OriginatorOrg: xilinx.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Nov 2020 07:48:46.2489 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 190ee4eb-4b9a-4f74-b159-08d895045e0d X-MS-Exchange-CrossTenant-Id: 657af505-d5df-48d0-8300-c31994686c5c X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=657af505-d5df-48d0-8300-c31994686c5c; Ip=[149.199.62.198]; Helo=[xsj-pvapexch01.xlnx.xilinx.com] X-MS-Exchange-CrossTenant-AuthSource: SN1NAM02FT051.eop-nam02.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN7PR02MB5041 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Nishad Saraf AI engine errors events can be routed to generate interrupt. The errors events routing will be done during AI engine configuration. At runtime, Linux kernel AI engine driver monitors the interrupt and backtracks errors events. As error events from 400 AIE tiles and 50 shim tiles are channeled on a single interrupt line, backtracking the source the interrupt to an AIE module is required. To keep the top-half interrupt short, backtracking is deferred to bottom half by scheduling a task in shared workqueue. Signed-off-by: Nishad Saraf Signed-off-by: Wendy Liang --- drivers/misc/xilinx-ai-engine/Makefile | 1 + drivers/misc/xilinx-ai-engine/ai-engine-aie.c | 121 ++++ drivers/misc/xilinx-ai-engine/ai-engine-dev.c | 14 + drivers/misc/xilinx-ai-engine/ai-engine-internal.h | 144 +++++ .../misc/xilinx-ai-engine/ai-engine-interrupt.c | 659 +++++++++++++++++++++ drivers/misc/xilinx-ai-engine/ai-engine-part.c | 44 ++ drivers/misc/xilinx-ai-engine/ai-engine-res.c | 54 ++ 7 files changed, 1037 insertions(+) create mode 100644 drivers/misc/xilinx-ai-engine/ai-engine-interrupt.c diff --git a/drivers/misc/xilinx-ai-engine/Makefile b/drivers/misc/xilinx-ai-engine/Makefile index 2e67b25..9607ecb 100644 --- a/drivers/misc/xilinx-ai-engine/Makefile +++ b/drivers/misc/xilinx-ai-engine/Makefile @@ -9,6 +9,7 @@ xilinx-aie-$(CONFIG_XILINX_AIE) := ai-engine-aie.o \ ai-engine-clock.o \ ai-engine-dev.o \ ai-engine-dma.o \ + ai-engine-interrupt.o \ ai-engine-mem.o \ ai-engine-part.o \ ai-engine-res.o \ diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-aie.c b/drivers/misc/xilinx-ai-engine/ai-engine-aie.c index ff721b3..af0f997 100644 --- a/drivers/misc/xilinx-ai-engine/ai-engine-aie.c +++ b/drivers/misc/xilinx-ai-engine/ai-engine-aie.c @@ -33,7 +33,10 @@ #define AIE_SHIMPL_CLKCNTR_REGOFF 0x00036040U #define AIE_SHIMPL_COLRESET_REGOFF 0x00036048U #define AIE_SHIMPL_RESET_REGOFF 0x0003604cU +#define AIE_SHIMPL_GROUP_ERROR_REGOFF 0x0003450cU #define AIE_TILE_CORE_CLKCNTR_REGOFF 0x00036040U +#define AIE_TILE_CORE_GROUP_ERROR_REGOFF 0x00034510U +#define AIE_TILE_MEM_GROUP_ERROR_REGOFF 0x00014514U /* * Register masks @@ -93,11 +96,27 @@ static const struct aie_tile_regs aie_kernel_regs[] = { .soff = AIE_SHIMPL_CLKCNTR_REGOFF, .eoff = AIE_SHIMPL_CLKCNTR_REGOFF, }, + /* SHIM group error enable */ + {.attribute = (AIE_TILE_TYPE_SHIMPL | AIE_TILE_TYPE_SHIMNOC) << + AIE_REGS_ATTR_TILE_TYPE_SHIFT, + .soff = AIE_SHIMPL_GROUP_ERROR_REGOFF, + .eoff = AIE_SHIMPL_GROUP_ERROR_REGOFF, + }, /* Tile clock control */ {.attribute = AIE_TILE_TYPE_TILE << AIE_REGS_ATTR_TILE_TYPE_SHIFT, .soff = AIE_TILE_CORE_CLKCNTR_REGOFF, .eoff = AIE_TILE_CORE_CLKCNTR_REGOFF, }, + /* Tile group error for core module */ + {.attribute = AIE_TILE_TYPE_TILE << AIE_REGS_ATTR_TILE_TYPE_SHIFT, + .soff = AIE_TILE_CORE_GROUP_ERROR_REGOFF, + .eoff = AIE_TILE_CORE_GROUP_ERROR_REGOFF, + }, + /* Tile group error for memory module */ + {.attribute = AIE_TILE_TYPE_TILE << AIE_REGS_ATTR_TILE_TYPE_SHIFT, + .soff = AIE_TILE_MEM_GROUP_ERROR_REGOFF, + .eoff = AIE_TILE_MEM_GROUP_ERROR_REGOFF, + }, }; static const struct aie_single_reg_field aie_col_rst = { @@ -128,6 +147,103 @@ static const struct aie_dma_attr aie_shimdma = { .bd_len = 0x14U, }; +static const struct aie_event_attr aie_pl_event = { + .bc_event = { + .mask = GENMASK(6, 0), + .regoff = 0x0U, + }, + .group_error = { + .mask = GENMASK(10, 0), + .regoff = 0xcU, + }, + .bc_regoff = 0x34010U, + .status_regoff = 0x34200U, + .group_regoff = 0x34500U, + .base_error_event = 62U, + .num_broadcasts = 16U, + .base_bc_event = 107U, + .num_events = 128U, +}; + +static const struct aie_event_attr aie_mem_event = { + .bc_event = { + .mask = GENMASK(6, 0), + .regoff = 0x0U, + }, + .group_error = { + .mask = GENMASK(13, 0), + .regoff = 0x14U, + }, + .bc_regoff = 0x14010U, + .status_regoff = 0x14200U, + .group_regoff = 0x14500U, + .base_error_event = 87U, + .num_broadcasts = 16U, + .base_bc_event = 107U, + .num_events = 128U, +}; + +static const struct aie_event_attr aie_core_event = { + .bc_event = { + .mask = GENMASK(6, 0), + .regoff = 0x0U, + }, + .group_error = { + .mask = GENMASK(21, 0), + .regoff = 0x10U, + }, + .bc_regoff = 0x34010U, + .status_regoff = 0x34200U, + .group_regoff = 0x34500U, + .base_error_event = 48U, + .num_broadcasts = 16U, + .base_bc_event = 107U, + .num_events = 128U, +}; + +static const struct aie_l1_intr_ctrl_attr aie_l1_intr_ctrl = { + .swa_status = { + .mask = GENMASK(19, 0), + .regoff = 0xcU, + }, + .swb_status = { + .mask = GENMASK(19, 0), + .regoff = 0x3cU, + }, + .swa_event = { + .mask = GENMASK(6, 0), + .regoff = 0x14U, + }, + .swb_event = { + .mask = GENMASK(6, 0), + .regoff = 0x44U, + }, + .regoff = 0x35000U, + .event_lsb = 8, + .num_broadcasts = 0x14U, +}; + +static const struct aie_l2_intr_ctrl_attr aie_l2_intr_ctrl = { + .mask = { + .mask = GENMASK(15, 0), + .regoff = 0x0U, + }, + .enable = { + .mask = GENMASK(15, 0), + .regoff = 0x4U, + }, + .disable = { + .mask = GENMASK(15, 0), + .regoff = 0x8U, + }, + .status = { + .mask = GENMASK(15, 0), + .regoff = 0xcU, + }, + .regoff = 0x15000U, + .num_broadcasts = 0x10U, +}; + static u32 aie_get_tile_type(struct aie_location *loc) { if (loc->row) @@ -476,6 +592,11 @@ int aie_device_init(struct aie_device *adev) adev->col_rst = &aie_col_rst; adev->col_clkbuf = &aie_col_clkbuf; adev->shim_dma = &aie_shimdma; + adev->pl_events = &aie_pl_event; + adev->mem_events = &aie_mem_event; + adev->core_events = &aie_core_event; + adev->l1_ctrl = &aie_l1_intr_ctrl; + adev->l2_ctrl = &aie_l2_intr_ctrl; /* Get the columns resource */ /* Get number of columns from AI engine memory resource */ diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-dev.c b/drivers/misc/xilinx-ai-engine/ai-engine-dev.c index 51c3a4f..240ff6c 100644 --- a/drivers/misc/xilinx-ai-engine/ai-engine-dev.c +++ b/drivers/misc/xilinx-ai-engine/ai-engine-dev.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -406,6 +407,19 @@ static int xilinx_ai_engine_probe(struct platform_device *pdev) of_xilinx_ai_engine_part_probe(adev); dev_info(&pdev->dev, "Xilinx AI Engine device(cols=%u) probed\n", adev->cols_res.total); + + INIT_WORK(&adev->backtrack, aie_array_backtrack); + + adev->irq = platform_get_irq_byname(pdev, "interrupt1"); + if (adev->irq < 0) + goto free_ida; + + ret = devm_request_threaded_irq(dev, adev->irq, NULL, aie_interrupt, + IRQF_ONESHOT, dev_name(dev), adev); + if (ret) { + dev_err(&pdev->dev, "Failed to request AIE IRQ.\n"); + goto free_ida; + } return 0; free_ida: diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-internal.h b/drivers/misc/xilinx-ai-engine/ai-engine-internal.h index b21b7025..3b680f7 100644 --- a/drivers/misc/xilinx-ai-engine/ai-engine-internal.h +++ b/drivers/misc/xilinx-ai-engine/ai-engine-internal.h @@ -15,6 +15,7 @@ #include #include #include +#include #include #include #include @@ -45,6 +46,55 @@ #define VERSAL_ES1_REV_ID 0x0 #define VERSAL_ES2_REV_ID 0x1 +#define AIE_NPI_ERROR_ID BIT(1) + +/* Macros relevant to interrupts */ +#define AIE_INTR_L2_CTRL_MASK_WIDTH 32 + +/** + * enum aie_module_type - identifies different hardware modules within a + * tile type. AIE tile may have memory and core + * module. While a PL or shim tile may have PL module. + * @AIE_MEM_MOD: comprises of the following sub-modules, + * * data memory. + * * tile DMA. + * * lock module. + * * events, event broadcast and event actions. + * * tracing and profiling. + * @AIE_CORE_MOD: comprises of the following sub-modules, + * * AIE core. + * * program Memory. + * * events, event broadcast and event actions. + * * tracing and profiling. + * * AXI-MM and AXI-S tile interconnects. + * @AIE_PL_MOD: comprises of the following sub-modules, + * * PL interface. + * * AXI-MM and AXI-S tile interconnects. + * * Level 1 interrupt controllers. + * * events, event broadcast and event actions. + * * tracing and profiling. + * @AIE_NOC_MOD: comprises of the following sub-modules, + * * interface from NoC Slave Unit (NSU) + * (bridge to AXI-MM switch) + * * interfaces to NoC NoC Master Unit (NMU) + * * shim DMA & locks + * * NoC stream interface + */ +enum aie_module_type { + AIE_MEM_MOD, + AIE_CORE_MOD, + AIE_PL_MOD, + AIE_NOC_MOD, +}; + +/* + * enum aie_shim_switch_type - identifies different switches in shim tile. + */ +enum aie_shim_switch_type { + AIE_SHIM_SWITCH_A, + AIE_SHIM_SWITCH_B +}; + /** * struct aie_tile_regs - contiguous range of AI engine register * within an AI engine tile @@ -157,6 +207,75 @@ struct aie_resource { }; /** + * struct aie_event_attr - AI Engine event attributes structure. + * @bc_event: broadcast event attribute to capture event mask value and + * register offset from @bc_regoff. + * @group_error: group error attribute to capture error group mask value and + * register offset value from @group_regoff. + * @bc_regoff: base broadcast register offset. + * @status_regoff: base status register offset. + * @group_regoff: base group error register offset. + * @base_error_event: event ID of first error event in a group error. + * @num_broadcasts: total number of broadcast events. + * @base_bc_event: broadcast 0 vent ID + * @num_events: total number of events. + */ +struct aie_event_attr { + struct aie_single_reg_field bc_event; + struct aie_single_reg_field group_error; + u32 bc_regoff; + u32 status_regoff; + u32 group_regoff; + u32 base_error_event; + u32 num_broadcasts; + u32 base_bc_event; + u32 num_events; +}; + +/** + * struct aie_l1_intr_ctrl_attr - AI engine level 1 interrupt controller + * attributes structure. + * @mask: level 1 interrupt controller mask attribute. + * @swa_status: switch A level 1 interrupt controller status attribute. + * @swb_status: switch A level 1 interrupt controller status attribute. + * @swa_event: switch A level 1 interrupt controller event attribute. + * @swb_event: switch A level 1 interrupt controller event attribute. + * @regoff: base level 1 interrupt controller register offset. + * @event_lsb: lsb of IRQ event within IRQ event switch register. + * @num_broadcasts: total number of broadcast signals to level 1 interrupt + * controller. + */ +struct aie_l1_intr_ctrl_attr { + struct aie_single_reg_field swa_status; + struct aie_single_reg_field swb_status; + struct aie_single_reg_field swa_event; + struct aie_single_reg_field swb_event; + u32 regoff; + u32 event_lsb; + u32 num_broadcasts; +}; + +/** + * struct aie_l2_intr_ctrl_attr - AI engine level 2 interrupt controller + * attributes structure. + * @mask: level 2 interrupt controller mask attribute. + * @enable: level 2 interrupt controller enable attribute. + * @disable: level 2 interrupt controller disable attribute. + * @status: level 2 interrupt controller status attribute. + * @regoff: level 2 interrupt controller register offset. + * @num_broadcasts: total number of broadcast signals to level 2 interrupt + * controller. + */ +struct aie_l2_intr_ctrl_attr { + struct aie_single_reg_field mask; + struct aie_single_reg_field enable; + struct aie_single_reg_field disable; + struct aie_single_reg_field status; + u32 regoff; + u32 num_broadcasts; +}; + +/** * struct aie_device - AI engine device structure * @partitions: list of partitions requested * @cdev: cdev for the AI engine @@ -169,6 +288,11 @@ struct aie_resource { * @col_rst: column reset attribute * @col_clkbuf: column clock buffer attribute * @shim_dma: SHIM DMA attribute + * @pl_events: pl module event attribute + * @mem_events: memory module event attribute + * @core_events: core module event attribute + * @l1_ctrl: level 1 interrupt controller attribute + * @l2_ctrl: level 2 interrupt controller attribute * @size: size of the AI engine address space * @array_shift: array address shift * @col_shift: column address shift @@ -176,6 +300,8 @@ struct aie_resource { * @cols_res: AI engine columns resources to indicate * while columns are occupied by partitions. * @num_kernel_regs: number of kernel only registers range + * @irq: Linux IRQ number + * @backtrack: workqueue to backtrack interrupt * @version: AI engine device version * @pm_node_id: AI Engine platform management node ID */ @@ -191,12 +317,19 @@ struct aie_device { const struct aie_single_reg_field *col_rst; const struct aie_single_reg_field *col_clkbuf; const struct aie_dma_attr *shim_dma; + const struct aie_event_attr *pl_events; + const struct aie_event_attr *mem_events; + const struct aie_event_attr *core_events; + const struct aie_l1_intr_ctrl_attr *l1_ctrl; + const struct aie_l2_intr_ctrl_attr *l2_ctrl; size_t size; struct aie_resource cols_res; u32 array_shift; u32 col_shift; u32 row_shift; u32 num_kernel_regs; + int irq; + struct work_struct backtrack; int version; u32 pm_node_id; }; @@ -212,6 +345,7 @@ struct aie_device { * @dev: device for the AI engine partition * @cores_clk_state: bitmap to indicate the power state of core modules * @tiles_inuse: bitmap to indicate if a tile is in use + * @l2_mask: level 2 interrupt controller mask bitmap * @partition_id: partition id. Partition ID is the identifier * of the AI engine partition in the system. * @status: indicate if the partition is in use @@ -228,6 +362,7 @@ struct aie_partition { struct device dev; struct aie_resource cores_clk_state; struct aie_resource tiles_inuse; + struct aie_resource l2_mask; u32 partition_id; u32 status; u32 cntrflag; @@ -339,7 +474,12 @@ int aie_resource_get_region(struct aie_resource *res, u32 start, void aie_resource_put_region(struct aie_resource *res, int start, u32 count); int aie_resource_set(struct aie_resource *res, u32 start, u32 count); int aie_resource_clear(struct aie_resource *res, u32 start, u32 count); +int aie_resource_clear_all(struct aie_resource *res); bool aie_resource_testbit(struct aie_resource *res, u32 bit); +int aie_resource_cpy_from_arr32(struct aie_resource *res, u32 start, + const u32 *src, u32 nbits); +int aie_resource_cpy_to_arr32(struct aie_resource *res, u32 start, u32 *dst, + u32 nbits); const struct file_operations *aie_part_get_fops(void); u8 aie_part_in_use(struct aie_partition *apart); @@ -372,4 +512,8 @@ int aie_part_release_tiles_from_user(struct aie_partition *apart, void __user *user_args); int aie_device_init(struct aie_device *adev); + +void aie_array_backtrack(struct work_struct *work); +irqreturn_t aie_interrupt(int irq, void *data); + #endif /* AIE_INTERNAL_H */ diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-interrupt.c b/drivers/misc/xilinx-ai-engine/ai-engine-interrupt.c new file mode 100644 index 0000000..c2fd4a0 --- /dev/null +++ b/drivers/misc/xilinx-ai-engine/ai-engine-interrupt.c @@ -0,0 +1,659 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Xilinx AI Engine device driver + * + * Copyright (C) 2020 Xilinx, Inc. + */ +#include +#include +#include +#include +#include +#include + +#include "ai-engine-internal.h" + +#define AIE_ARRAY_TILE_ERROR_BC_ID 0U +#define AIE_SHIM_TILE_ERROR_IRQ_ID 16U + +/** + * aie_get_broadcast_event() - get event ID being broadcast on given + * broadcast line. + * @apart: AIE partition pointer. + * @loc: pointer to tile location. + * @module: module type. + * @bc_id: broadcast ID. + * @return: event ID. + */ +static u8 aie_get_broadcast_event(struct aie_partition *apart, + struct aie_location *loc, + enum aie_module_type module, u8 bc_id) +{ + const struct aie_event_attr *event_mod; + u32 bcoff, regoff; + + if (module == AIE_CORE_MOD) + event_mod = apart->adev->core_events; + else if (module == AIE_MEM_MOD) + event_mod = apart->adev->mem_events; + else + event_mod = apart->adev->pl_events; + + bcoff = event_mod->bc_regoff + event_mod->bc_event.regoff + bc_id * 4U; + regoff = aie_cal_regoff(apart->adev, *loc, bcoff); + return ioread32(apart->adev->base + regoff); +} + +/** + * aie_read_event_status() - get the status of event status registers. + * @apart: AIE partition pointer. + * @loc: pointer to tile location. + * @module: module type. + * @reg: array to store event status register values. + */ +static void aie_read_event_status(struct aie_partition *apart, + struct aie_location *loc, + enum aie_module_type module, u32 *reg) +{ + const struct aie_event_attr *event_mod; + u8 offset; + + if (module == AIE_CORE_MOD) + event_mod = apart->adev->core_events; + else if (module == AIE_MEM_MOD) + event_mod = apart->adev->mem_events; + else + event_mod = apart->adev->pl_events; + + for (offset = 0; offset < (event_mod->num_events / 32); offset++) { + u32 status_off = event_mod->status_regoff + offset * 4U; + u32 regoff = aie_cal_regoff(apart->adev, *loc, status_off); + + reg[offset] = ioread32(apart->adev->base + regoff); + } +} + +/** + * aie_check_group_errors_enabled() - get error events enabled in group error. + * @apart: AIE partition pointer. + * @loc: pointer to tile location. + * @module: module type. + * @return: bitmap of enabled error events. + */ +static u32 aie_check_group_errors_enabled(struct aie_partition *apart, + struct aie_location *loc, + enum aie_module_type module) +{ + const struct aie_event_attr *event_mod; + u32 groff, regoff; + + if (module == AIE_CORE_MOD) + event_mod = apart->adev->core_events; + else if (module == AIE_MEM_MOD) + event_mod = apart->adev->mem_events; + else + event_mod = apart->adev->pl_events; + + groff = event_mod->group_regoff + event_mod->group_error.regoff; + regoff = aie_cal_regoff(apart->adev, *loc, groff); + return ioread32(apart->adev->base + regoff); +} + +/** + * aie_set_error_event() - enable/disable error events in group error. + * @apart: AIE partition pointer. + * @loc: pointer to tile location. + * @module: module type. + * @bitmap: error event to enable/disable in group errors. + */ +static void aie_set_error_event(struct aie_partition *apart, + struct aie_location *loc, + enum aie_module_type module, u32 bitmap) +{ + const struct aie_event_attr *event_mod; + u32 groff, regoff; + + if (module == AIE_CORE_MOD) + event_mod = apart->adev->core_events; + else if (module == AIE_MEM_MOD) + event_mod = apart->adev->mem_events; + else + event_mod = apart->adev->pl_events; + + groff = event_mod->group_regoff + event_mod->group_error.regoff; + regoff = aie_cal_regoff(apart->adev, *loc, groff); + iowrite32(bitmap, apart->adev->base + regoff); +} + +/** + * aie_get_error_event() - map group error status bit to actual error + * event number. + * @apart: AIE partition pointer. + * @loc: pointer to tile location. + * @module: module type. + * @index: event index within group errors. + * @return: true event ID. + */ +static u32 aie_get_error_event(struct aie_partition *apart, + struct aie_location *loc, + enum aie_module_type module, u8 index) +{ + const struct aie_event_attr *event_mod; + + if (module == AIE_CORE_MOD) + event_mod = apart->adev->core_events; + else if (module == AIE_MEM_MOD) + event_mod = apart->adev->mem_events; + else + event_mod = apart->adev->pl_events; + + return event_mod->base_error_event + index; +} + +/** + * aie_get_bc_event() - get the broadcast event ID. + * @apart: AIE partition pointer. + * @loc: pointer to tile location. + * @module: module type. + * @bc_id: broadcast line ID. + * @return: broadcast event ID. + */ +static u32 aie_get_bc_event(struct aie_partition *apart, + struct aie_location *loc, + enum aie_module_type module, u8 bc_id) +{ + const struct aie_event_attr *event_mod; + + if (module == AIE_CORE_MOD) + event_mod = apart->adev->core_events; + else if (module == AIE_MEM_MOD) + event_mod = apart->adev->mem_events; + else + event_mod = apart->adev->pl_events; + + return event_mod->base_bc_event + bc_id; +} + +/** + * aie_get_l1_event() - get event ID being broadcast on level 1 IRQ. + * @apart: AIE partition pointer. + * @loc: pointer to tile location. + * @sw: switch type. + * @irq_id: IRQ event ID to be read. + * @return: true event ID. + */ +static u8 aie_get_l1_event(struct aie_partition *apart, + struct aie_location *loc, + enum aie_shim_switch_type sw, u8 irq_id) +{ + const struct aie_l1_intr_ctrl_attr *intr_ctrl = apart->adev->l1_ctrl; + u32 l1off, l1mask, regoff, reg_value; + + if (sw == AIE_SHIM_SWITCH_A) { + l1off = intr_ctrl->regoff + intr_ctrl->swa_event.regoff; + l1mask = intr_ctrl->swa_event.mask; + } else { + l1off = intr_ctrl->regoff + intr_ctrl->swb_event.regoff; + l1mask = intr_ctrl->swb_event.mask; + } + + regoff = aie_cal_regoff(apart->adev, *loc, l1off); + reg_value = ioread32(apart->adev->base + regoff); + reg_value &= l1mask << (irq_id * intr_ctrl->event_lsb); + reg_value >>= (irq_id * intr_ctrl->event_lsb); + return reg_value; +} + +/** + * aie_clear_l1_intr() - clear level 1 interrupt controller status. + * @apart: AIE partition pointer. + * @loc: pointer to tile location. + * @sw: switch type. + * @irq_id: IRQ ID to be cleared. + */ +static void aie_clear_l1_intr(struct aie_partition *apart, + struct aie_location *loc, + enum aie_shim_switch_type sw, u8 irq_id) +{ + const struct aie_l1_intr_ctrl_attr *intr_ctrl = apart->adev->l1_ctrl; + u32 l1off, regoff; + + if (sw == AIE_SHIM_SWITCH_A) + l1off = intr_ctrl->regoff + intr_ctrl->swa_status.regoff; + else + l1off = intr_ctrl->regoff + intr_ctrl->swb_status.regoff; + + regoff = aie_cal_regoff(apart->adev, *loc, l1off); + iowrite32(BIT(irq_id), apart->adev->base + regoff); +} + +/** + * aie_get_l1_status() - get level 1 interrupt controller status value. + * @apart: AIE partition pointer. + * @loc: pointer to tile location. + * @sw: switch type. + * @return: status value. + */ +static u32 aie_get_l1_status(struct aie_partition *apart, + struct aie_location *loc, + enum aie_shim_switch_type sw) +{ + const struct aie_l1_intr_ctrl_attr *intr_ctrl = apart->adev->l1_ctrl; + u32 l1off, regoff; + + if (sw == AIE_SHIM_SWITCH_A) + l1off = intr_ctrl->regoff + intr_ctrl->swa_status.regoff; + else + l1off = intr_ctrl->regoff + intr_ctrl->swb_status.regoff; + + regoff = aie_cal_regoff(apart->adev, *loc, l1off); + return ioread32(apart->adev->base + regoff); +} + +/** + * aie_clear_l2_intr() - clear level 2 interrupt controller status. + * @apart: AIE partition pointer. + * @loc: pointer to tile location. + * @bitmap_irq: IRQ bitmap. IRQ lines corresponding to set bits will be + * cleared. + */ +static void aie_clear_l2_intr(struct aie_partition *apart, + struct aie_location *loc, u32 bitmap_irq) +{ + const struct aie_l2_intr_ctrl_attr *intr_ctrl = apart->adev->l2_ctrl; + u32 l2off = intr_ctrl->regoff + intr_ctrl->status.regoff; + u32 regoff = aie_cal_regoff(apart->adev, *loc, l2off); + + iowrite32(bitmap_irq, apart->adev->base + regoff); +} + +/** + * aie_get_l2_status() - get level 2 interrupt controller status value. + * @apart: AIE partition pointer. + * @loc: pointer to tile location. + * @return: status value. + */ +static u32 aie_get_l2_status(struct aie_partition *apart, + struct aie_location *loc) +{ + const struct aie_l2_intr_ctrl_attr *intr_ctrl = apart->adev->l2_ctrl; + u32 l2off = intr_ctrl->regoff + intr_ctrl->status.regoff; + u32 regoff = aie_cal_regoff(apart->adev, *loc, l2off); + + return ioread32(apart->adev->base + regoff); +} + +/** + * aie_get_l2_mask() - get level 2 interrupt controller mask value. + * @apart: AIE partition pointer. + * @loc: pointer to tile location. + * @return: mask value. + */ +static u32 aie_get_l2_mask(struct aie_partition *apart, + struct aie_location *loc) +{ + const struct aie_l2_intr_ctrl_attr *intr_ctrl = apart->adev->l2_ctrl; + u32 l2off = intr_ctrl->regoff + intr_ctrl->mask.regoff; + u32 regoff = aie_cal_regoff(apart->adev, *loc, l2off); + + return ioread32(apart->adev->base + regoff); +} + +/** + * aie_enable_l2_ctrl() - enable interrupts to level 2 interrupt controller. + * @apart: AIE partition pointer. + * @loc: pointer to tile location. + * @bit_map: bitmap of broadcast lines to enable. + */ +static void aie_enable_l2_ctrl(struct aie_partition *apart, + struct aie_location *loc, u32 bit_map) +{ + const struct aie_l2_intr_ctrl_attr *intr_ctrl = apart->adev->l2_ctrl; + u32 l2off = intr_ctrl->regoff + intr_ctrl->enable.regoff; + u32 regoff = aie_cal_regoff(apart->adev, *loc, l2off); + + bit_map &= intr_ctrl->enable.mask; + iowrite32(bit_map, apart->adev->base + regoff); +} + +/** + * aie_disable_l2_ctrl() - disable interrupts to level 2 interrupt controller. + * @apart: AIE partition pointer. + * @loc: pointer to tile location. + * @bit_map: bitmap of broadcast lines to disable. + */ +static void aie_disable_l2_ctrl(struct aie_partition *apart, + struct aie_location *loc, u32 bit_map) +{ + const struct aie_l2_intr_ctrl_attr *intr_ctrl = apart->adev->l2_ctrl; + u32 l2off = intr_ctrl->regoff + intr_ctrl->disable.regoff; + u32 regoff = aie_cal_regoff(apart->adev, *loc, l2off); + + bit_map &= intr_ctrl->disable.mask; + iowrite32(bit_map, apart->adev->base + regoff); +} + +/** + * aie_tile_backtrack() - if error was asserted on a broadcast line in + * the given array tile, + * * disable the error from the group errors + * @apart: AIE partition pointer. + * @loc: tile location. + * @module: module type. + * @sw: switch type. + * @bc_id: broadcast ID. + * @return: true if error was asserted, else return false. + */ +static bool aie_tile_backtrack(struct aie_partition *apart, + struct aie_location loc, + enum aie_module_type module, + enum aie_shim_switch_type sw, u8 bc_id) +{ + unsigned long grenabled; + u32 status[4]; + u8 n, grevent, eevent; + bool ret = false; + + if (module == AIE_PL_MOD) + grevent = aie_get_l1_event(apart, &loc, sw, bc_id); + else + grevent = aie_get_broadcast_event(apart, &loc, module, bc_id); + + aie_read_event_status(apart, &loc, module, status); + + if (!(status[grevent / 32] & BIT(grevent % 32))) + return ret; + + grenabled = aie_check_group_errors_enabled(apart, &loc, module); + for_each_set_bit(n, &grenabled, 32) { + eevent = aie_get_error_event(apart, &loc, module, n); + if (!(status[eevent / 32] & BIT(eevent % 32))) + continue; + grenabled &= ~BIT(n); + ret = true; + dev_err_ratelimited(&apart->adev->dev, + "Asserted tile error event %d at col %d row %d\n", + eevent, loc.col, loc.row); + } + aie_set_error_event(apart, &loc, module, grenabled); + + return ret; +} + +/** + * aie_map_l2_to_l1() - map the status bit set in level 2 interrupt controller + * to a level 1 interrupt controller. + * @apart: AIE partition pointer. + * @set_pos: position of level 2 set bit. + * @l2_col: level 2 interrupt controller column ID. + * @l1_col: pointer to return corresponding level 1 column ID. + * @sw: pointer to return the level 1 interrupt controller switch ID. + * + * This API implementation is tightly coupled with the level 2 to level 1 + * static mapping created when AIE application CDOs are generated. + */ +static void aie_map_l2_to_l1(struct aie_partition *apart, u32 set_pos, + u32 l2_col, u32 *l1_col, + enum aie_shim_switch_type *sw) +{ + if (l2_col + 3 >= apart->range.start.col + apart->range.size.col) { + *l1_col = l2_col + (set_pos % 6) / 2; + *sw = (set_pos % 6) % 2; + } else if (l2_col % 2 == 0) { + /* set bit position could be 0 - 5 */ + *l1_col = l2_col - (2 - (set_pos % 6) / 2); + *sw = (set_pos % 6) % 2; + } else { + /* set bit position could be 0 - 1 */ + *l1_col = l2_col; + *sw = set_pos; + } +} + +/** + * aie_l1_backtrack() - backtrack AIE array tiles or shim tile based on + * the level 2 status bit set. + * @apart: AIE partition pointer. + * @loc: tile location of level 2 interrupt controller. + * @set_pos: set bit position in level 2 controller status. + * @return: true if error was asserted, else return false. + */ +static bool aie_l1_backtrack(struct aie_partition *apart, + struct aie_location loc, u32 set_pos) +{ + struct aie_location l1_ctrl; + enum aie_shim_switch_type sw; + unsigned long status; + u32 srow = apart->range.start.row + 1; + u32 erow = apart->range.start.row + apart->range.size.row; + bool ret = false; + + /* + * Based on the set status bit find which level 1 interrupt + * controller has generated an interrupt + */ + l1_ctrl.row = 0; + aie_map_l2_to_l1(apart, set_pos, loc.col, &l1_ctrl.col, &sw); + + status = aie_get_l1_status(apart, &l1_ctrl, sw); + + /* For now, support error broadcasts only */ + if (status & BIT(AIE_ARRAY_TILE_ERROR_BC_ID)) { + struct aie_location temp; + enum aie_module_type module; + u32 bc_event; + + if (sw == AIE_SHIM_SWITCH_A) + module = AIE_CORE_MOD; + else + module = AIE_MEM_MOD; + + aie_clear_l1_intr(apart, &l1_ctrl, sw, + AIE_ARRAY_TILE_ERROR_BC_ID); + + temp.row = srow; + temp.col = l1_ctrl.col; + bc_event = aie_get_bc_event(apart, &temp, module, + AIE_ARRAY_TILE_ERROR_BC_ID); + for (; temp.row < erow; temp.row++) { + u32 reg[4]; + + if (!aie_part_check_clk_enable_loc(apart, &temp)) + break; + + if (aie_tile_backtrack(apart, temp, module, sw, + AIE_ARRAY_TILE_ERROR_BC_ID)) + ret = true; + + aie_read_event_status(apart, &temp, module, reg); + if (!(reg[bc_event / 32] & BIT(bc_event % 32))) + break; + } + } + + if (status & BIT(AIE_SHIM_TILE_ERROR_IRQ_ID)) { + aie_clear_l1_intr(apart, &l1_ctrl, sw, + AIE_SHIM_TILE_ERROR_IRQ_ID); + if (aie_tile_backtrack(apart, l1_ctrl, AIE_PL_MOD, sw, + AIE_SHIM_TILE_ERROR_IRQ_ID)) + ret = true; + } + return ret; +} + +/** + * aie_l2_backtrack() - iterate through each level 2 interrupt controller + * in a given partition and backtrack its + * corresponding level 1 interrupt controller. + * @apart: AIE partition pointer + */ +static void aie_l2_backtrack(struct aie_partition *apart) +{ + struct aie_location loc; + unsigned long l2_mask = 0; + u32 n, ttype, l2_bitmap_offset = 0; + int ret; + bool sched_work = false; + + ret = mutex_lock_interruptible(&apart->mlock); + if (ret) { + dev_err_ratelimited(&apart->dev, + "Failed to acquire lock. Process was interrupted by fatal signals\n"); + return; + } + + for (loc.col = apart->range.start.col, loc.row = 0; + loc.col < apart->range.start.col + apart->range.size.col; + loc.col++) { + ttype = apart->adev->ops->get_tile_type(&loc); + if (ttype != AIE_TILE_TYPE_SHIMNOC) + continue; + + aie_resource_cpy_to_arr32(&apart->l2_mask, l2_bitmap_offset * + 32, (u32 *)&l2_mask, 32); + l2_bitmap_offset++; + + for_each_set_bit(n, &l2_mask, + apart->adev->l2_ctrl->num_broadcasts) + aie_l1_backtrack(apart, loc, n); + + aie_enable_l2_ctrl(apart, &loc, l2_mask); + } + + /* + * Level 2 interrupt registers are edge-triggered. As a result, + * re-enabling level 2 won't trigger an interrupt for the already + * latched interrupts at level 1 controller. + */ + for (loc.col = apart->range.start.col, loc.row = 0; + loc.col < apart->range.start.col + apart->range.size.col; + loc.col++) { + if (aie_get_l1_status(apart, &loc, AIE_SHIM_SWITCH_A) || + aie_get_l1_status(apart, &loc, AIE_SHIM_SWITCH_B)) { + mutex_unlock(&apart->mlock); + sched_work = true; + schedule_work(&apart->adev->backtrack); + break; + } + } + + if (!sched_work) + mutex_unlock(&apart->mlock); +} + +/** + * aie_part_backtrack() - backtrack a individual. + * @apart: AIE partition pointer. + */ +static void aie_part_backtrack(struct aie_partition *apart) +{ + aie_l2_backtrack(apart); +} + +/** + * aie_array_backtrack() - backtrack each partition to find the source of error + * interrupt. + * @work: pointer to the work structure. + * + * This task will re-enable IRQ after errors in all partitions has been + * serviced. + */ +void aie_array_backtrack(struct work_struct *work) +{ + struct aie_device *adev; + struct aie_partition *apart; + int ret; + + adev = container_of(work, struct aie_device, backtrack); + + ret = mutex_lock_interruptible(&adev->mlock); + if (ret) { + dev_err_ratelimited(&adev->dev, + "Failed to acquire lock. Process was interrupted by fatal signals\n"); + return; + } + + list_for_each_entry(apart, &adev->partitions, node) + aie_part_backtrack(apart); + + mutex_unlock(&adev->mlock); +} + +/** + * aie_interrupt() - interrupt handler for AIE. + * @irq: Interrupt number. + * @data: AI engine device structure. + * @return: IRQ_HANDLED. + * + * This thread function disables level 2 interrupt controllers and schedules a + * task in workqueue to backtrack the source of error interrupt. Disabled + * interrupts are re-enabled after successful completion of bottom half. + */ +irqreturn_t aie_interrupt(int irq, void *data) +{ + struct aie_device *adev = data; + struct aie_partition *apart; + int ret; + bool sched_work = false; + + ret = mutex_lock_interruptible(&adev->mlock); + if (ret) { + dev_err(&adev->dev, + "Failed to acquire lock. Process was interrupted by fatal signals\n"); + return IRQ_NONE; + } + + list_for_each_entry(apart, &adev->partitions, node) { + struct aie_location loc; + u32 ttype, l2_mask, l2_status, l2_bitmap_offset = 0; + + ret = mutex_lock_interruptible(&apart->mlock); + if (ret) { + dev_err(&apart->dev, + "Failed to acquire lock. Process was interrupted by fatal signals\n"); + mutex_unlock(&adev->mlock); + return IRQ_NONE; + } + + for (loc.col = apart->range.start.col, loc.row = 0; + loc.col < apart->range.start.col + apart->range.size.col; + loc.col++) { + ttype = apart->adev->ops->get_tile_type(&loc); + if (ttype != AIE_TILE_TYPE_SHIMNOC) + continue; + + l2_mask = aie_get_l2_mask(apart, &loc); + if (l2_mask) { + aie_resource_cpy_from_arr32(&apart->l2_mask, + l2_bitmap_offset * + 32, &l2_mask, 32); + aie_disable_l2_ctrl(apart, &loc, l2_mask); + } + l2_bitmap_offset++; + + l2_status = aie_get_l2_status(apart, &loc); + if (l2_status) { + aie_clear_l2_intr(apart, &loc, l2_status); + sched_work = true; + } else { + aie_enable_l2_ctrl(apart, &loc, l2_mask); + } + } + mutex_unlock(&apart->mlock); + } + + /* For ES1 silicon, interrupts are latched in NPI */ + if (adev->version == VERSAL_ES1_REV_ID) { + ret = zynqmp_pm_clear_aie_npi_isr(adev->pm_node_id, + AIE_NPI_ERROR_ID); + if (ret < 0) + dev_err(&adev->dev, "Failed to clear NPI ISR\n"); + } + + mutex_unlock(&adev->mlock); + + if (sched_work) + schedule_work(&adev->backtrack); + + return IRQ_HANDLED; +} diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-part.c b/drivers/misc/xilinx-ai-engine/ai-engine-part.c index 54450b6..0670d0ad 100644 --- a/drivers/misc/xilinx-ai-engine/ai-engine-part.c +++ b/drivers/misc/xilinx-ai-engine/ai-engine-part.c @@ -247,6 +247,9 @@ static int aie_part_release(struct inode *inode, struct file *filp) aie_part_clean(apart); apart->status = 0; + + aie_resource_clear_all(&apart->l2_mask); + mutex_unlock(&apart->mlock); return 0; @@ -369,6 +372,7 @@ static void aie_part_release_device(struct device *dev) list_del(&apart->node); mutex_unlock(&adev->mlock); aie_resource_uninitialize(&apart->cores_clk_state); + aie_resource_uninitialize(&apart->l2_mask); put_device(apart->dev.parent); } @@ -408,6 +412,39 @@ static int aie_part_create_mems_info(struct aie_partition *apart) } /** + * aie_part_create_l2_bitmap() - create bitmaps to record mask and status + * values for level 2 interrupt controllers. + * @apart: AI engine partition + * @return: 0 for success, and negative value for failure. + */ +static int aie_part_create_l2_bitmap(struct aie_partition *apart) +{ + struct aie_location loc; + u8 num_l2_ctrls = 0; + int ret; + + loc.row = 0; + for (loc.col = apart->range.start.col; + loc.col < apart->range.start.col + apart->range.size.col; + loc.col++) { + u32 ttype = apart->adev->ops->get_tile_type(&loc); + + if (ttype == AIE_TILE_TYPE_SHIMNOC) + num_l2_ctrls++; + } + + ret = aie_resource_initialize(&apart->l2_mask, num_l2_ctrls * + AIE_INTR_L2_CTRL_MASK_WIDTH); + if (ret) { + dev_err(&apart->dev, + "failed to initialize l2 mask resource.\n"); + return ret; + } + + return 0; +} + +/** * aie_create_partition() - create AI engine partition instance * @adev: AI engine device * @range: AI engine partition range to check. A range describes a group @@ -498,6 +535,13 @@ static struct aie_partition *aie_create_partition(struct aie_device *adev, return ERR_PTR(ret); } + ret = aie_part_create_l2_bitmap(apart); + if (ret < 0) { + dev_err(&apart->dev, "Failed to allocate l2 bitmap.\n"); + put_device(dev); + return ERR_PTR(ret); + } + ret = mutex_lock_interruptible(&adev->mlock); if (ret) { put_device(dev); diff --git a/drivers/misc/xilinx-ai-engine/ai-engine-res.c b/drivers/misc/xilinx-ai-engine/ai-engine-res.c index b0c0741..f1bb75b 100644 --- a/drivers/misc/xilinx-ai-engine/ai-engine-res.c +++ b/drivers/misc/xilinx-ai-engine/ai-engine-res.c @@ -132,6 +132,44 @@ int aie_resource_set(struct aie_resource *res, u32 start, u32 count) } /** + * aie_resource_cpy_from_arr32() - copies nbits from u32[] to bitmap. + * @res: pointer to AI engine resource + * @start: start bit in bitmap + * @src: source buffer + * @nbits: number of bits to copy from u32[] + * @return: 0 for success and negative value for failure + */ +int aie_resource_cpy_from_arr32(struct aie_resource *res, u32 start, + const u32 *src, u32 nbits) +{ + if (!res || !res->bitmap || !nbits || start + nbits > res->total || + !src) + return -EINVAL; + + bitmap_from_arr32(res->bitmap + BIT_WORD(start), src, nbits); + return 0; +} + +/** + * aie_resource_cpy_to_arr32() - copies nbits to u32[] from bitmap. + * @res: pointer to AI engine resource + * @start: start bit in bitmap + * @dst: destination buffer + * @nbits: number of bits to copy to u32[] + * @return: 0 for success and negative value for failure + */ +int aie_resource_cpy_to_arr32(struct aie_resource *res, u32 start, u32 *dst, + u32 nbits) +{ + if (!res || !res->bitmap || !nbits || start + nbits > res->total || + !dst) + return -EINVAL; + + bitmap_to_arr32(dst, res->bitmap + BIT_WORD(start), nbits); + return 0; +} + +/** * aie_resource_clear() - clear the AI engine resource bits * @res: pointer to AI engine resource * @start: start bit to set @@ -150,6 +188,22 @@ int aie_resource_clear(struct aie_resource *res, u32 start, u32 count) } /** + * aie_resource_clear_all() - clear all the AI engine resource bits + * @res: pointer to AI engine resource + * @return: 0 for success and negative value for failure + * + * This function clears all the bits in the resource. + */ +int aie_resource_clear_all(struct aie_resource *res) +{ + if (!res || !res->bitmap) + return -EINVAL; + + bitmap_clear(res->bitmap, 0, res->total); + return 0; +} + +/** * aie_resource_testbit() - test if a bit is set in a AI engine resource * @res: pointer to AI engine resource * @bit: bit to check