From patchwork Mon Dec 18 11:31:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755796 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1F5D51BDE1; Mon, 18 Dec 2023 11:33:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="FrZmTNPP" Received: from pps.filterd (m0279871.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAsPSk000317; Mon, 18 Dec 2023 11:33:10 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=uemSpET3sD+B6z4KBIkwMzg9w2z6sbdfcQ+wwx6z580=; b=Fr ZmTNPPqEeQ0fk1Dx9CSOl3m7BN4zZy5fEqiTGZN0Z/0uBjd48/qUEmDJYG6V8q6K 9mqoHJ75IQCz/QQX+ju4IQ68OrnonYzp9cBZRJSXSjJ6w8tdFmsRvAB/c3bZWrYp H75Yz6C2xCVLNtyLaWrxrQXirD3YW4guBQwIOnN0oH1NkOEt3/V2+70CS6+raXvg CUOi5rPom2vCFqfdD8yhM1Alxy7g1DWLDqVPL7t1uoObiThFg1DBcLaeJIh/LKW9 RCixp3UXAk6NdY82FQZ0voYpzH+A7qvNB+HxoGgNUOEOxDNCLkTu3fhObW+zUxOG Wm6/piFOe2OFmVtxE1Sg== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v14xy443u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:33:09 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5MB029943; Mon, 18 Dec 2023 11:33:05 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyg9-1; Mon, 18 Dec 2023 11:33:05 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5Gj029914; Mon, 18 Dec 2023 11:33:05 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX53B029905; Mon, 18 Dec 2023 11:33:05 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 211F0A34; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 01/34] media: introduce common helpers for video firmware handling Date: Mon, 18 Dec 2023 17:01:56 +0530 Message-Id: <1702899149-21321-2-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: x3Oy-UIFpQzs5vsfJ35JZG1ksAM8in_R X-Proofpoint-ORIG-GUID: x3Oy-UIFpQzs5vsfJ35JZG1ksAM8in_R X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 malwarescore=0 phishscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 impostorscore=0 lowpriorityscore=0 suspectscore=0 spamscore=0 bulkscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Re-organize the video driver code by introducing a new folder 'vcodec' and placing 'venus' driver code inside that. Introduce common helpers for trustzone based firmware load/unload etc. which are placed in common folder i.e. 'vcodec'. Use these helpers in 'venus' driver. These helpers will be used by 'iris' driver as well which is introduced later in this patch series. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/Kconfig | 2 +- drivers/media/platform/qcom/Makefile | 2 +- drivers/media/platform/qcom/vcodec/firmware.c | 147 +++++++++ drivers/media/platform/qcom/vcodec/firmware.h | 21 ++ .../media/platform/qcom/{ => vcodec}/venus/Kconfig | 0 .../platform/qcom/{ => vcodec}/venus/Makefile | 4 +- .../media/platform/qcom/{ => vcodec}/venus/core.c | 102 +++++- .../media/platform/qcom/{ => vcodec}/venus/core.h | 0 .../media/platform/qcom/{ => vcodec}/venus/dbgfs.c | 0 .../media/platform/qcom/{ => vcodec}/venus/dbgfs.h | 0 .../platform/qcom/vcodec/venus/firmware_no_tz.c | 194 +++++++++++ .../platform/qcom/vcodec/venus/firmware_no_tz.h | 19 ++ .../platform/qcom/{ => vcodec}/venus/helpers.c | 0 .../platform/qcom/{ => vcodec}/venus/helpers.h | 0 .../media/platform/qcom/{ => vcodec}/venus/hfi.c | 0 .../media/platform/qcom/{ => vcodec}/venus/hfi.h | 0 .../platform/qcom/{ => vcodec}/venus/hfi_cmds.c | 0 .../platform/qcom/{ => vcodec}/venus/hfi_cmds.h | 0 .../platform/qcom/{ => vcodec}/venus/hfi_helper.h | 0 .../platform/qcom/{ => vcodec}/venus/hfi_msgs.c | 0 .../platform/qcom/{ => vcodec}/venus/hfi_msgs.h | 0 .../platform/qcom/{ => vcodec}/venus/hfi_parser.c | 0 .../platform/qcom/{ => vcodec}/venus/hfi_parser.h | 0 .../qcom/{ => vcodec}/venus/hfi_plat_bufs.h | 0 .../qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c | 0 .../qcom/{ => vcodec}/venus/hfi_platform.c | 0 .../qcom/{ => vcodec}/venus/hfi_platform.h | 0 .../qcom/{ => vcodec}/venus/hfi_platform_v4.c | 0 .../qcom/{ => vcodec}/venus/hfi_platform_v6.c | 0 .../platform/qcom/{ => vcodec}/venus/hfi_venus.c | 21 +- .../platform/qcom/{ => vcodec}/venus/hfi_venus.h | 0 .../qcom/{ => vcodec}/venus/hfi_venus_io.h | 0 .../platform/qcom/{ => vcodec}/venus/pm_helpers.c | 0 .../platform/qcom/{ => vcodec}/venus/pm_helpers.h | 0 .../media/platform/qcom/{ => vcodec}/venus/vdec.c | 0 .../media/platform/qcom/{ => vcodec}/venus/vdec.h | 0 .../platform/qcom/{ => vcodec}/venus/vdec_ctrls.c | 0 .../media/platform/qcom/{ => vcodec}/venus/venc.c | 0 .../media/platform/qcom/{ => vcodec}/venus/venc.h | 0 .../platform/qcom/{ => vcodec}/venus/venc_ctrls.c | 0 drivers/media/platform/qcom/venus/firmware.c | 363 --------------------- drivers/media/platform/qcom/venus/firmware.h | 26 -- 42 files changed, 492 insertions(+), 409 deletions(-) create mode 100644 drivers/media/platform/qcom/vcodec/firmware.c create mode 100644 drivers/media/platform/qcom/vcodec/firmware.h rename drivers/media/platform/qcom/{ => vcodec}/venus/Kconfig (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/Makefile (83%) rename drivers/media/platform/qcom/{ => vcodec}/venus/core.c (91%) rename drivers/media/platform/qcom/{ => vcodec}/venus/core.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/dbgfs.h (100%) create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c create mode 100644 drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/helpers.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_cmds.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_helper.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_msgs.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_parser.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_plat_bufs_v6.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v4.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_platform_v6.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.c (99%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/hfi_venus_io.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/pm_helpers.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/vdec_ctrls.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.c (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/venc.h (100%) rename drivers/media/platform/qcom/{ => vcodec}/venus/venc_ctrls.c (100%) delete mode 100644 drivers/media/platform/qcom/venus/firmware.c delete mode 100644 drivers/media/platform/qcom/venus/firmware.h diff --git a/drivers/media/platform/qcom/Kconfig b/drivers/media/platform/qcom/Kconfig index cc5799b..e94142f 100644 --- a/drivers/media/platform/qcom/Kconfig +++ b/drivers/media/platform/qcom/Kconfig @@ -3,4 +3,4 @@ comment "Qualcomm media platform drivers" source "drivers/media/platform/qcom/camss/Kconfig" -source "drivers/media/platform/qcom/venus/Kconfig" +source "drivers/media/platform/qcom/vcodec/venus/Kconfig" diff --git a/drivers/media/platform/qcom/Makefile b/drivers/media/platform/qcom/Makefile index 4f055c3..3d2d82b 100644 --- a/drivers/media/platform/qcom/Makefile +++ b/drivers/media/platform/qcom/Makefile @@ -1,3 +1,3 @@ # SPDX-License-Identifier: GPL-2.0-only obj-y += camss/ -obj-y += venus/ +obj-y += vcodec/venus/ diff --git a/drivers/media/platform/qcom/vcodec/firmware.c b/drivers/media/platform/qcom/vcodec/firmware.c new file mode 100644 index 0000000..dbc220a --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/firmware.c @@ -0,0 +1,147 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "firmware.h" + +bool use_tz(struct device *core_dev) +{ + struct device_node *np; + + np = of_get_child_by_name(core_dev->of_node, "video-firmware"); + if (!np) + return true; + + return false; +} + +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start, + u32 cp_nonpixel_size, u32 pas_id) +{ + int ret; + /* + * Clues for porting using downstream data: + * cp_start = 0 + * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size! + * This works, as the non-secure context bank is placed + * contiguously right after the Content Protection region. + * + * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0] + * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1] + */ + ret = qcom_scm_mem_protect_video_var(cp_start, + cp_size, + cp_nonpixel_start, + cp_nonpixel_size); + if (ret) + qcom_scm_pas_shutdown(pas_id); + + return ret; +} + +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys, + size_t *mem_size, u32 pas_id, bool use_tz) +{ + const struct firmware *firmware = NULL; + struct reserved_mem *rmem; + struct device_node *node; + void *mem_virt = NULL; + ssize_t fw_size = 0; + int ret; + + if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) || + (use_tz && !qcom_scm_is_available())) + return -EPROBE_DEFER; + + if (!fw_name || !(*fw_name)) + return -EINVAL; + + *mem_phys = 0; + *mem_size = 0; + + node = of_parse_phandle(dev->of_node, "memory-region", 0); + if (!node) { + dev_err(dev, "no memory-region specified\n"); + return -EINVAL; + } + + rmem = of_reserved_mem_lookup(node); + of_node_put(node); + if (!rmem) { + dev_err(dev, "failed to lookup reserved memory-region\n"); + return -EINVAL; + } + + ret = request_firmware(&firmware, fw_name, dev); + if (ret) { + dev_err(dev, "%s: failed to request fw \"%s\", error %d\n", + __func__, fw_name, ret); + return ret; + } + + fw_size = qcom_mdt_get_size(firmware); + if (fw_size < 0) { + ret = fw_size; + dev_err(dev, "%s: out of bound fw image fw size: %ld\n", + __func__, fw_size); + goto err_release_fw; + } + + *mem_phys = rmem->base; + *mem_size = rmem->size; + + if (*mem_size < fw_size) { + ret = -EINVAL; + goto err_release_fw; + } + + mem_virt = memremap(*mem_phys, *mem_size, MEMREMAP_WC); + if (!mem_virt) { + dev_err(dev, "unable to remap fw memory region %pa size %#zx\n", + mem_phys, *mem_size); + goto err_release_fw; + } + + if (use_tz) + ret = qcom_mdt_load(dev, firmware, fw_name, pas_id, mem_virt, + *mem_phys, *mem_size, NULL); + else + ret = qcom_mdt_load_no_init(dev, firmware, fw_name, pas_id, mem_virt, + *mem_phys, *mem_size, NULL); + if (ret) { + dev_err(dev, "%s: error %d loading fw \"%s\"\n", + __func__, ret, fw_name); + } + + memunmap(mem_virt); +err_release_fw: + release_firmware(firmware); + return ret; +} + +int auth_reset_fw(u32 pas_id) +{ + return qcom_scm_pas_auth_and_reset(pas_id); +} + +void unload_fw(u32 pas_id) +{ + qcom_scm_pas_shutdown(pas_id); +} + +int set_hw_state(bool resume) +{ + return qcom_scm_set_remote_state(resume, 0); +} diff --git a/drivers/media/platform/qcom/vcodec/firmware.h b/drivers/media/platform/qcom/vcodec/firmware.h new file mode 100644 index 0000000..7d410a8 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/firmware.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _FIRMWARE_H_ +#define _FIRMWARE_H_ + +#include +#include + +bool use_tz(struct device *core_dev); +int load_fw(struct device *dev, const char *fw_name, phys_addr_t *mem_phys, + size_t *mem_size, u32 pas_id, bool use_tz); +int auth_reset_fw(u32 pas_id); +int protect_secure_region(u32 cp_start, u32 cp_size, u32 cp_nonpixel_start, + u32 cp_nonpixel_size, u32 pas_id); +void unload_fw(u32 pas_id); +int set_hw_state(bool resume); + +#endif diff --git a/drivers/media/platform/qcom/venus/Kconfig b/drivers/media/platform/qcom/vcodec/venus/Kconfig similarity index 100% rename from drivers/media/platform/qcom/venus/Kconfig rename to drivers/media/platform/qcom/vcodec/venus/Kconfig diff --git a/drivers/media/platform/qcom/venus/Makefile b/drivers/media/platform/qcom/vcodec/venus/Makefile similarity index 83% rename from drivers/media/platform/qcom/venus/Makefile rename to drivers/media/platform/qcom/vcodec/venus/Makefile index 91ee6be..f6f3a88 100644 --- a/drivers/media/platform/qcom/venus/Makefile +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile @@ -1,7 +1,9 @@ # SPDX-License-Identifier: GPL-2.0 # Makefile for Qualcomm Venus driver -venus-core-objs += core.o helpers.o firmware.o \ +venus-core-objs += ../firmware.o + +venus-core-objs += core.o helpers.o firmware_no_tz.o \ hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \ hfi_parser.o pm_helpers.o dbgfs.o \ hfi_platform.o hfi_platform_v4.o \ diff --git a/drivers/media/platform/qcom/venus/core.c b/drivers/media/platform/qcom/vcodec/venus/core.c similarity index 91% rename from drivers/media/platform/qcom/venus/core.c rename to drivers/media/platform/qcom/vcodec/venus/core.c index 9cffe97..56d9a53 100644 --- a/drivers/media/platform/qcom/venus/core.c +++ b/drivers/media/platform/qcom/vcodec/venus/core.c @@ -22,7 +22,8 @@ #include #include "core.h" -#include "firmware.h" +#include "../firmware.h" +#include "firmware_no_tz.h" #include "pm_helpers.h" #include "hfi_venus_io.h" @@ -86,6 +87,8 @@ static void venus_sys_error_handler(struct work_struct *work) struct venus_core *core = container_of(work, struct venus_core, work.work); int ret, i, max_attempts = RPM_WAIT_FOR_IDLE_MAX_ATTEMPTS; + const struct venus_resources *res = core->res; + const char *fwpath = NULL; const char *err_msg = ""; bool failed = false; @@ -107,7 +110,10 @@ static void venus_sys_error_handler(struct work_struct *work) mutex_lock(&core->lock); - venus_shutdown(core); + if (core->use_tz) + unload_fw(VENUS_PAS_ID); + else + unload_fw_no_tz(core); venus_coredump(core); @@ -127,12 +133,39 @@ static void venus_sys_error_handler(struct work_struct *work) failed = true; } - ret = venus_boot(core); + ret = of_property_read_string_index(core->dev->of_node, "firmware-name", 0, + &fwpath); + if (ret) + fwpath = core->res->fwname; + + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys, &core->fw.mem_size, + VENUS_PAS_ID, core->use_tz); if (ret && !failed) { - err_msg = "boot Venus"; + err_msg = "load FW"; failed = true; } + if (core->use_tz) + ret = auth_reset_fw(VENUS_PAS_ID); + else + ret = auth_reset_fw_no_tz(core, core->fw.mem_phys, core->fw.mem_size); + if (ret && !failed) { + err_msg = "Auth and Reset"; + failed = true; + } + + if (core->use_tz && res->cp_size) { + ret = protect_secure_region(res->cp_start, + res->cp_size, + res->cp_nonpixel_start, + res->cp_nonpixel_size, + VENUS_PAS_ID); + if (ret && !failed) { + err_msg = "Protect CP Mem"; + failed = true; + } + } + ret = hfi_core_resume(core, true); if (ret && !failed) { err_msg = "resume HFI"; @@ -281,7 +314,9 @@ static irqreturn_t venus_isr_thread(int irq, void *dev_id) static int venus_probe(struct platform_device *pdev) { + const struct venus_resources *res; struct device *dev = &pdev->dev; + const char *fwpath = NULL; struct venus_core *core; int ret; @@ -362,14 +397,42 @@ static int venus_probe(struct platform_device *pdev) if (ret) goto err_runtime_disable; - ret = venus_firmware_init(core); + core->use_tz = use_tz(core->dev); + + if (!core->use_tz) { + ret = init_fw_no_tz(core); + if (ret) + goto err_of_depopulate; + } + + ret = of_property_read_string_index(dev->of_node, "firmware-name", 0, + &fwpath); if (ret) - goto err_of_depopulate; + fwpath = core->res->fwname; - ret = venus_boot(core); + ret = load_fw(core->dev, fwpath, &core->fw.mem_phys, &core->fw.mem_size, + VENUS_PAS_ID, core->use_tz); if (ret) goto err_firmware_deinit; + if (core->use_tz) + ret = auth_reset_fw(VENUS_PAS_ID); + else + ret = auth_reset_fw_no_tz(core, core->fw.mem_phys, core->fw.mem_size); + if (ret) + goto err_firmware_deinit; + + res = core->res; + if (core->use_tz && res->cp_size) { + ret = protect_secure_region(res->cp_start, + res->cp_size, + res->cp_nonpixel_start, + res->cp_nonpixel_size, + VENUS_PAS_ID); + if (ret) + goto err_firmware_deinit; + } + ret = hfi_core_resume(core, true); if (ret) goto err_venus_shutdown; @@ -399,9 +462,13 @@ static int venus_probe(struct platform_device *pdev) err_dev_unregister: v4l2_device_unregister(&core->v4l2_dev); err_venus_shutdown: - venus_shutdown(core); + if (core->use_tz) + unload_fw(VENUS_PAS_ID); + else + unload_fw_no_tz(core); err_firmware_deinit: - venus_firmware_deinit(core); + if (!core->use_tz) + deinit_fw_no_tz(core); err_of_depopulate: of_platform_depopulate(dev); err_runtime_disable: @@ -430,10 +497,15 @@ static void venus_remove(struct platform_device *pdev) ret = hfi_core_deinit(core, true); WARN_ON(ret); - venus_shutdown(core); + if (core->use_tz) + unload_fw(VENUS_PAS_ID); + else + unload_fw_no_tz(core); + of_platform_depopulate(dev); - venus_firmware_deinit(core); + if (!core->use_tz) + deinit_fw_no_tz(core); pm_runtime_put_sync(dev); pm_runtime_disable(dev); @@ -455,8 +527,12 @@ static void venus_core_shutdown(struct platform_device *pdev) struct venus_core *core = platform_get_drvdata(pdev); pm_runtime_get_sync(core->dev); - venus_shutdown(core); - venus_firmware_deinit(core); + if (core->use_tz) { + unload_fw(VENUS_PAS_ID); + } else { + unload_fw_no_tz(core); + deinit_fw_no_tz(core); + } pm_runtime_put_sync(core->dev); } diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/vcodec/venus/core.h similarity index 100% rename from drivers/media/platform/qcom/venus/core.h rename to drivers/media/platform/qcom/vcodec/venus/core.h diff --git a/drivers/media/platform/qcom/venus/dbgfs.c b/drivers/media/platform/qcom/vcodec/venus/dbgfs.c similarity index 100% rename from drivers/media/platform/qcom/venus/dbgfs.c rename to drivers/media/platform/qcom/vcodec/venus/dbgfs.c diff --git a/drivers/media/platform/qcom/venus/dbgfs.h b/drivers/media/platform/qcom/vcodec/venus/dbgfs.h similarity index 100% rename from drivers/media/platform/qcom/venus/dbgfs.h rename to drivers/media/platform/qcom/vcodec/venus/dbgfs.h diff --git a/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c new file mode 100644 index 0000000..9dca6e23 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.c @@ -0,0 +1,194 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2017 Linaro Ltd. + */ + +#include +#include +#include "core.h" +#include "firmware_no_tz.h" +#include "hfi_venus_io.h" + +#define VENUS_FW_MEM_SIZE (6 * SZ_1M) +#define VENUS_FW_START_ADDR 0x0 + +int init_fw_no_tz(struct venus_core *core) +{ + struct platform_device_info info; + struct iommu_domain *iommu_dom; + struct platform_device *pdev; + struct device_node *np; + int ret; + + np = of_get_child_by_name(core->dev->of_node, "video-firmware"); + + memset(&info, 0, sizeof(info)); + info.fwnode = &np->fwnode; + info.parent = core->dev; + info.name = np->name; + info.dma_mask = DMA_BIT_MASK(32); + + pdev = platform_device_register_full(&info); + if (IS_ERR(pdev)) { + of_node_put(np); + return PTR_ERR(pdev); + } + + pdev->dev.of_node = np; + + ret = of_dma_configure(&pdev->dev, np, true); + if (ret) { + dev_err(core->dev, "dma configure fail\n"); + goto err_unregister; + } + + core->fw.dev = &pdev->dev; + + iommu_dom = iommu_domain_alloc(&platform_bus_type); + if (!iommu_dom) { + dev_err(core->fw.dev, "Failed to allocate iommu domain\n"); + ret = -ENOMEM; + goto err_unregister; + } + + ret = iommu_attach_device(iommu_dom, core->fw.dev); + if (ret) { + dev_err(core->fw.dev, "could not attach device\n"); + goto err_iommu_free; + } + + core->fw.iommu_domain = iommu_dom; + + of_node_put(np); + + return 0; + +err_iommu_free: + iommu_domain_free(iommu_dom); +err_unregister: + platform_device_unregister(pdev); + of_node_put(np); + return ret; +} + +void deinit_fw_no_tz(struct venus_core *core) +{ + struct iommu_domain *iommu; + + if (!core->fw.dev) + return; + + iommu = core->fw.iommu_domain; + + iommu_detach_device(iommu, core->fw.dev); + + if (iommu) { + iommu_domain_free(iommu); + iommu = NULL; + } + + platform_device_unregister(to_platform_device(core->fw.dev)); +} + +static void reset_cpu_no_tz(struct venus_core *core) +{ + u32 fw_size = core->fw.mapped_mem_size; + void __iomem *wrapper_base; + + if (IS_IRIS2_1(core)) + wrapper_base = core->wrapper_tz_base; + else + wrapper_base = core->wrapper_base; + + writel(0, wrapper_base + WRAPPER_FW_START_ADDR); + writel(fw_size, wrapper_base + WRAPPER_FW_END_ADDR); + writel(0, wrapper_base + WRAPPER_CPA_START_ADDR); + writel(fw_size, wrapper_base + WRAPPER_CPA_END_ADDR); + writel(fw_size, wrapper_base + WRAPPER_NONPIX_START_ADDR); + writel(fw_size, wrapper_base + WRAPPER_NONPIX_END_ADDR); + + if (IS_IRIS2_1(core)) { + /* Bring XTSS out of reset */ + writel(0, wrapper_base + WRAPPER_TZ_XTSS_SW_RESET); + } else { + writel(0x0, wrapper_base + WRAPPER_CPU_CGC_DIS); + writel(0x0, wrapper_base + WRAPPER_CPU_CLOCK_CONFIG); + + /* Bring ARM9 out of reset */ + writel(0, wrapper_base + WRAPPER_A9SS_SW_RESET); + } +} + +void set_hw_state_no_tz(struct venus_core *core, bool resume) +{ + if (resume) { + reset_cpu_no_tz(core); + } else { + if (IS_IRIS2_1(core)) + writel(WRAPPER_XTSS_SW_RESET_BIT, + core->wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET); + else + writel(WRAPPER_A9SS_SW_RESET_BIT, + core->wrapper_base + WRAPPER_A9SS_SW_RESET); + } +} + +int auth_reset_fw_no_tz(struct venus_core *core, phys_addr_t mem_phys, + size_t mem_size) +{ + struct iommu_domain *iommu; + struct device *dev; + int ret; + + dev = core->fw.dev; + if (!dev) + return -EPROBE_DEFER; + + iommu = core->fw.iommu_domain; + core->fw.mapped_mem_size = mem_size; + + ret = iommu_map(iommu, VENUS_FW_START_ADDR, mem_phys, mem_size, + IOMMU_READ | IOMMU_WRITE | IOMMU_PRIV, GFP_KERNEL); + if (ret) { + dev_err(dev, "could not map video firmware region\n"); + return ret; + } + + reset_cpu_no_tz(core); + + return 0; +} + +void unload_fw_no_tz(struct venus_core *core) +{ + const size_t mapped = core->fw.mapped_mem_size; + struct iommu_domain *iommu; + size_t unmapped; + u32 reg; + struct device *dev = core->fw.dev; + void __iomem *wrapper_base = core->wrapper_base; + void __iomem *wrapper_tz_base = core->wrapper_tz_base; + + if (IS_IRIS2_1(core)) { + /* Assert the reset to XTSS */ + reg = readl(wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET); + reg |= WRAPPER_XTSS_SW_RESET_BIT; + writel(reg, wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET); + } else { + /* Assert the reset to ARM9 */ + reg = readl(wrapper_base + WRAPPER_A9SS_SW_RESET); + reg |= WRAPPER_A9SS_SW_RESET_BIT; + writel(reg, wrapper_base + WRAPPER_A9SS_SW_RESET); + } + + iommu = core->fw.iommu_domain; + + if (core->fw.mapped_mem_size && iommu) { + unmapped = iommu_unmap(iommu, VENUS_FW_START_ADDR, mapped); + + if (unmapped != mapped) + dev_err(dev, "failed to unmap firmware\n"); + else + core->fw.mapped_mem_size = 0; + } +} diff --git a/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h new file mode 100644 index 0000000..5f008ef --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/venus/firmware_no_tz.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2017 Linaro Ltd. + */ +#ifndef __FIRMWARE_NO_TZ_H__ +#define __FIRMWARE_NO_TZ_H__ + +struct device; + +#define VENUS_PAS_ID 9 + +int init_fw_no_tz(struct venus_core *core); +void deinit_fw_no_tz(struct venus_core *core); +int auth_reset_fw_no_tz(struct venus_core *core, phys_addr_t mem_phys, + size_t mem_size); +void unload_fw_no_tz(struct venus_core *core); +void set_hw_state_no_tz(struct venus_core *core, bool resume); + +#endif diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/vcodec/venus/helpers.c similarity index 100% rename from drivers/media/platform/qcom/venus/helpers.c rename to drivers/media/platform/qcom/vcodec/venus/helpers.c diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/vcodec/venus/helpers.h similarity index 100% rename from drivers/media/platform/qcom/venus/helpers.h rename to drivers/media/platform/qcom/vcodec/venus/helpers.h diff --git a/drivers/media/platform/qcom/venus/hfi.c b/drivers/media/platform/qcom/vcodec/venus/hfi.c similarity index 100% rename from drivers/media/platform/qcom/venus/hfi.c rename to drivers/media/platform/qcom/vcodec/venus/hfi.c diff --git a/drivers/media/platform/qcom/venus/hfi.h b/drivers/media/platform/qcom/vcodec/venus/hfi.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi.h rename to drivers/media/platform/qcom/vcodec/venus/hfi.h diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.c b/drivers/media/platform/qcom/vcodec/venus/hfi_cmds.c similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_cmds.c rename to drivers/media/platform/qcom/vcodec/venus/hfi_cmds.c diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.h b/drivers/media/platform/qcom/vcodec/venus/hfi_cmds.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_cmds.h rename to drivers/media/platform/qcom/vcodec/venus/hfi_cmds.h diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h b/drivers/media/platform/qcom/vcodec/venus/hfi_helper.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_helper.h rename to drivers/media/platform/qcom/vcodec/venus/hfi_helper.h diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.c b/drivers/media/platform/qcom/vcodec/venus/hfi_msgs.c similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_msgs.c rename to drivers/media/platform/qcom/vcodec/venus/hfi_msgs.c diff --git a/drivers/media/platform/qcom/venus/hfi_msgs.h b/drivers/media/platform/qcom/vcodec/venus/hfi_msgs.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_msgs.h rename to drivers/media/platform/qcom/vcodec/venus/hfi_msgs.h diff --git a/drivers/media/platform/qcom/venus/hfi_parser.c b/drivers/media/platform/qcom/vcodec/venus/hfi_parser.c similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_parser.c rename to drivers/media/platform/qcom/vcodec/venus/hfi_parser.c diff --git a/drivers/media/platform/qcom/venus/hfi_parser.h b/drivers/media/platform/qcom/vcodec/venus/hfi_parser.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_parser.h rename to drivers/media/platform/qcom/vcodec/venus/hfi_parser.h diff --git a/drivers/media/platform/qcom/venus/hfi_plat_bufs.h b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_plat_bufs.h rename to drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs.h diff --git a/drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c b/drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c rename to drivers/media/platform/qcom/vcodec/venus/hfi_plat_bufs_v6.c diff --git a/drivers/media/platform/qcom/venus/hfi_platform.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform.c similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_platform.c rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform.c diff --git a/drivers/media/platform/qcom/venus/hfi_platform.h b/drivers/media/platform/qcom/vcodec/venus/hfi_platform.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_platform.h rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform.h diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v4.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform_v4.c similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_platform_v4.c rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform_v4.c diff --git a/drivers/media/platform/qcom/venus/hfi_platform_v6.c b/drivers/media/platform/qcom/vcodec/venus/hfi_platform_v6.c similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_platform_v6.c rename to drivers/media/platform/qcom/vcodec/venus/hfi_platform_v6.c diff --git a/drivers/media/platform/qcom/venus/hfi_venus.c b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c similarity index 99% rename from drivers/media/platform/qcom/venus/hfi_venus.c rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus.c index f9437b6..5a68db9 100644 --- a/drivers/media/platform/qcom/venus/hfi_venus.c +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c @@ -13,11 +13,12 @@ #include #include "core.h" +#include "../firmware.h" #include "hfi_cmds.h" #include "hfi_msgs.h" #include "hfi_venus.h" #include "hfi_venus_io.h" -#include "firmware.h" +#include "firmware_no_tz.h" #define HFI_MASK_QHDR_TX_TYPE 0xff000000 #define HFI_MASK_QHDR_RX_TYPE 0x00ff0000 @@ -635,7 +636,10 @@ static int venus_power_off(struct venus_hfi_device *hdev) if (!hdev->power_enabled) return 0; - ret = venus_set_hw_state_suspend(hdev->core); + if (hdev->core->use_tz) + ret = set_hw_state(false); + else + set_hw_state_no_tz(hdev->core, false); if (ret) return ret; @@ -655,7 +659,13 @@ static int venus_power_on(struct venus_hfi_device *hdev) if (hdev->power_enabled) return 0; - ret = venus_set_hw_state_resume(hdev->core); + if (hdev->core->use_tz) { + ret = set_hw_state(true); + if (ret == -EINVAL) + ret = 0; + } else { + set_hw_state_no_tz(hdev->core, true); + } if (ret) goto err; @@ -668,7 +678,10 @@ static int venus_power_on(struct venus_hfi_device *hdev) return 0; err_suspend: - venus_set_hw_state_suspend(hdev->core); + if (hdev->core->use_tz) + set_hw_state(false); + else + set_hw_state_no_tz(hdev->core, false); err: hdev->power_enabled = false; return ret; diff --git a/drivers/media/platform/qcom/venus/hfi_venus.h b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_venus.h rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus.h diff --git a/drivers/media/platform/qcom/venus/hfi_venus_io.h b/drivers/media/platform/qcom/vcodec/venus/hfi_venus_io.h similarity index 100% rename from drivers/media/platform/qcom/venus/hfi_venus_io.h rename to drivers/media/platform/qcom/vcodec/venus/hfi_venus_io.h diff --git a/drivers/media/platform/qcom/venus/pm_helpers.c b/drivers/media/platform/qcom/vcodec/venus/pm_helpers.c similarity index 100% rename from drivers/media/platform/qcom/venus/pm_helpers.c rename to drivers/media/platform/qcom/vcodec/venus/pm_helpers.c diff --git a/drivers/media/platform/qcom/venus/pm_helpers.h b/drivers/media/platform/qcom/vcodec/venus/pm_helpers.h similarity index 100% rename from drivers/media/platform/qcom/venus/pm_helpers.h rename to drivers/media/platform/qcom/vcodec/venus/pm_helpers.h diff --git a/drivers/media/platform/qcom/venus/vdec.c b/drivers/media/platform/qcom/vcodec/venus/vdec.c similarity index 100% rename from drivers/media/platform/qcom/venus/vdec.c rename to drivers/media/platform/qcom/vcodec/venus/vdec.c diff --git a/drivers/media/platform/qcom/venus/vdec.h b/drivers/media/platform/qcom/vcodec/venus/vdec.h similarity index 100% rename from drivers/media/platform/qcom/venus/vdec.h rename to drivers/media/platform/qcom/vcodec/venus/vdec.h diff --git a/drivers/media/platform/qcom/venus/vdec_ctrls.c b/drivers/media/platform/qcom/vcodec/venus/vdec_ctrls.c similarity index 100% rename from drivers/media/platform/qcom/venus/vdec_ctrls.c rename to drivers/media/platform/qcom/vcodec/venus/vdec_ctrls.c diff --git a/drivers/media/platform/qcom/venus/venc.c b/drivers/media/platform/qcom/vcodec/venus/venc.c similarity index 100% rename from drivers/media/platform/qcom/venus/venc.c rename to drivers/media/platform/qcom/vcodec/venus/venc.c diff --git a/drivers/media/platform/qcom/venus/venc.h b/drivers/media/platform/qcom/vcodec/venus/venc.h similarity index 100% rename from drivers/media/platform/qcom/venus/venc.h rename to drivers/media/platform/qcom/vcodec/venus/venc.h diff --git a/drivers/media/platform/qcom/venus/venc_ctrls.c b/drivers/media/platform/qcom/vcodec/venus/venc_ctrls.c similarity index 100% rename from drivers/media/platform/qcom/venus/venc_ctrls.c rename to drivers/media/platform/qcom/vcodec/venus/venc_ctrls.c diff --git a/drivers/media/platform/qcom/venus/firmware.c b/drivers/media/platform/qcom/venus/firmware.c deleted file mode 100644 index fe7da2b..0000000 --- a/drivers/media/platform/qcom/venus/firmware.c +++ /dev/null @@ -1,363 +0,0 @@ -// SPDX-License-Identifier: GPL-2.0-only -/* - * Copyright (C) 2017 Linaro Ltd. - */ - -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -#include "core.h" -#include "firmware.h" -#include "hfi_venus_io.h" - -#define VENUS_PAS_ID 9 -#define VENUS_FW_MEM_SIZE (6 * SZ_1M) -#define VENUS_FW_START_ADDR 0x0 - -static void venus_reset_cpu(struct venus_core *core) -{ - u32 fw_size = core->fw.mapped_mem_size; - void __iomem *wrapper_base; - - if (IS_IRIS2_1(core)) - wrapper_base = core->wrapper_tz_base; - else - wrapper_base = core->wrapper_base; - - writel(0, wrapper_base + WRAPPER_FW_START_ADDR); - writel(fw_size, wrapper_base + WRAPPER_FW_END_ADDR); - writel(0, wrapper_base + WRAPPER_CPA_START_ADDR); - writel(fw_size, wrapper_base + WRAPPER_CPA_END_ADDR); - writel(fw_size, wrapper_base + WRAPPER_NONPIX_START_ADDR); - writel(fw_size, wrapper_base + WRAPPER_NONPIX_END_ADDR); - - if (IS_IRIS2_1(core)) { - /* Bring XTSS out of reset */ - writel(0, wrapper_base + WRAPPER_TZ_XTSS_SW_RESET); - } else { - writel(0x0, wrapper_base + WRAPPER_CPU_CGC_DIS); - writel(0x0, wrapper_base + WRAPPER_CPU_CLOCK_CONFIG); - - /* Bring ARM9 out of reset */ - writel(0, wrapper_base + WRAPPER_A9SS_SW_RESET); - } -} - -int venus_set_hw_state(struct venus_core *core, bool resume) -{ - int ret; - - if (core->use_tz) { - ret = qcom_scm_set_remote_state(resume, 0); - if (resume && ret == -EINVAL) - ret = 0; - return ret; - } - - if (resume) { - venus_reset_cpu(core); - } else { - if (IS_IRIS2_1(core)) - writel(WRAPPER_XTSS_SW_RESET_BIT, - core->wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET); - else - writel(WRAPPER_A9SS_SW_RESET_BIT, - core->wrapper_base + WRAPPER_A9SS_SW_RESET); - } - - return 0; -} - -static int venus_load_fw(struct venus_core *core, const char *fwname, - phys_addr_t *mem_phys, size_t *mem_size) -{ - const struct firmware *mdt; - struct reserved_mem *rmem; - struct device_node *node; - struct device *dev; - ssize_t fw_size; - void *mem_va; - int ret; - - *mem_phys = 0; - *mem_size = 0; - - dev = core->dev; - node = of_parse_phandle(dev->of_node, "memory-region", 0); - if (!node) { - dev_err(dev, "no memory-region specified\n"); - return -EINVAL; - } - - rmem = of_reserved_mem_lookup(node); - of_node_put(node); - if (!rmem) { - dev_err(dev, "failed to lookup reserved memory-region\n"); - return -EINVAL; - } - - ret = request_firmware(&mdt, fwname, dev); - if (ret < 0) - return ret; - - fw_size = qcom_mdt_get_size(mdt); - if (fw_size < 0) { - ret = fw_size; - goto err_release_fw; - } - - *mem_phys = rmem->base; - *mem_size = rmem->size; - - if (*mem_size < fw_size || fw_size > VENUS_FW_MEM_SIZE) { - ret = -EINVAL; - goto err_release_fw; - } - - mem_va = memremap(*mem_phys, *mem_size, MEMREMAP_WC); - if (!mem_va) { - dev_err(dev, "unable to map memory region %pa size %#zx\n", mem_phys, *mem_size); - ret = -ENOMEM; - goto err_release_fw; - } - - if (core->use_tz) - ret = qcom_mdt_load(dev, mdt, fwname, VENUS_PAS_ID, - mem_va, *mem_phys, *mem_size, NULL); - else - ret = qcom_mdt_load_no_init(dev, mdt, fwname, VENUS_PAS_ID, - mem_va, *mem_phys, *mem_size, NULL); - - memunmap(mem_va); -err_release_fw: - release_firmware(mdt); - return ret; -} - -static int venus_boot_no_tz(struct venus_core *core, phys_addr_t mem_phys, - size_t mem_size) -{ - struct iommu_domain *iommu; - struct device *dev; - int ret; - - dev = core->fw.dev; - if (!dev) - return -EPROBE_DEFER; - - iommu = core->fw.iommu_domain; - core->fw.mapped_mem_size = mem_size; - - ret = iommu_map(iommu, VENUS_FW_START_ADDR, mem_phys, mem_size, - IOMMU_READ | IOMMU_WRITE | IOMMU_PRIV, GFP_KERNEL); - if (ret) { - dev_err(dev, "could not map video firmware region\n"); - return ret; - } - - venus_reset_cpu(core); - - return 0; -} - -static int venus_shutdown_no_tz(struct venus_core *core) -{ - const size_t mapped = core->fw.mapped_mem_size; - struct iommu_domain *iommu; - size_t unmapped; - u32 reg; - struct device *dev = core->fw.dev; - void __iomem *wrapper_base = core->wrapper_base; - void __iomem *wrapper_tz_base = core->wrapper_tz_base; - - if (IS_IRIS2_1(core)) { - /* Assert the reset to XTSS */ - reg = readl(wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET); - reg |= WRAPPER_XTSS_SW_RESET_BIT; - writel(reg, wrapper_tz_base + WRAPPER_TZ_XTSS_SW_RESET); - } else { - /* Assert the reset to ARM9 */ - reg = readl(wrapper_base + WRAPPER_A9SS_SW_RESET); - reg |= WRAPPER_A9SS_SW_RESET_BIT; - writel(reg, wrapper_base + WRAPPER_A9SS_SW_RESET); - } - - iommu = core->fw.iommu_domain; - - if (core->fw.mapped_mem_size && iommu) { - unmapped = iommu_unmap(iommu, VENUS_FW_START_ADDR, mapped); - - if (unmapped != mapped) - dev_err(dev, "failed to unmap firmware\n"); - else - core->fw.mapped_mem_size = 0; - } - - return 0; -} - -int venus_boot(struct venus_core *core) -{ - struct device *dev = core->dev; - const struct venus_resources *res = core->res; - const char *fwpath = NULL; - phys_addr_t mem_phys; - size_t mem_size; - int ret; - - if (!IS_ENABLED(CONFIG_QCOM_MDT_LOADER) || - (core->use_tz && !qcom_scm_is_available())) - return -EPROBE_DEFER; - - ret = of_property_read_string_index(dev->of_node, "firmware-name", 0, - &fwpath); - if (ret) - fwpath = core->res->fwname; - - ret = venus_load_fw(core, fwpath, &mem_phys, &mem_size); - if (ret) { - dev_err(dev, "fail to load video firmware\n"); - return -EINVAL; - } - - core->fw.mem_size = mem_size; - core->fw.mem_phys = mem_phys; - - if (core->use_tz) - ret = qcom_scm_pas_auth_and_reset(VENUS_PAS_ID); - else - ret = venus_boot_no_tz(core, mem_phys, mem_size); - - if (ret) - return ret; - - if (core->use_tz && res->cp_size) { - /* - * Clues for porting using downstream data: - * cp_start = 0 - * cp_size = venus_ns/virtual-addr-pool[0] - yes, address and not size! - * This works, as the non-secure context bank is placed - * contiguously right after the Content Protection region. - * - * cp_nonpixel_start = venus_sec_non_pixel/virtual-addr-pool[0] - * cp_nonpixel_size = venus_sec_non_pixel/virtual-addr-pool[1] - */ - ret = qcom_scm_mem_protect_video_var(res->cp_start, - res->cp_size, - res->cp_nonpixel_start, - res->cp_nonpixel_size); - if (ret) { - qcom_scm_pas_shutdown(VENUS_PAS_ID); - dev_err(dev, "set virtual address ranges fail (%d)\n", - ret); - return ret; - } - } - - return 0; -} - -int venus_shutdown(struct venus_core *core) -{ - int ret; - - if (core->use_tz) - ret = qcom_scm_pas_shutdown(VENUS_PAS_ID); - else - ret = venus_shutdown_no_tz(core); - - return ret; -} - -int venus_firmware_init(struct venus_core *core) -{ - struct platform_device_info info; - struct iommu_domain *iommu_dom; - struct platform_device *pdev; - struct device_node *np; - int ret; - - np = of_get_child_by_name(core->dev->of_node, "video-firmware"); - if (!np) { - core->use_tz = true; - return 0; - } - - memset(&info, 0, sizeof(info)); - info.fwnode = &np->fwnode; - info.parent = core->dev; - info.name = np->name; - info.dma_mask = DMA_BIT_MASK(32); - - pdev = platform_device_register_full(&info); - if (IS_ERR(pdev)) { - of_node_put(np); - return PTR_ERR(pdev); - } - - pdev->dev.of_node = np; - - ret = of_dma_configure(&pdev->dev, np, true); - if (ret) { - dev_err(core->dev, "dma configure fail\n"); - goto err_unregister; - } - - core->fw.dev = &pdev->dev; - - iommu_dom = iommu_domain_alloc(&platform_bus_type); - if (!iommu_dom) { - dev_err(core->fw.dev, "Failed to allocate iommu domain\n"); - ret = -ENOMEM; - goto err_unregister; - } - - ret = iommu_attach_device(iommu_dom, core->fw.dev); - if (ret) { - dev_err(core->fw.dev, "could not attach device\n"); - goto err_iommu_free; - } - - core->fw.iommu_domain = iommu_dom; - - of_node_put(np); - - return 0; - -err_iommu_free: - iommu_domain_free(iommu_dom); -err_unregister: - platform_device_unregister(pdev); - of_node_put(np); - return ret; -} - -void venus_firmware_deinit(struct venus_core *core) -{ - struct iommu_domain *iommu; - - if (!core->fw.dev) - return; - - iommu = core->fw.iommu_domain; - - iommu_detach_device(iommu, core->fw.dev); - - if (core->fw.iommu_domain) { - iommu_domain_free(iommu); - core->fw.iommu_domain = NULL; - } - - platform_device_unregister(to_platform_device(core->fw.dev)); -} diff --git a/drivers/media/platform/qcom/venus/firmware.h b/drivers/media/platform/qcom/venus/firmware.h deleted file mode 100644 index aaccd84..0000000 --- a/drivers/media/platform/qcom/venus/firmware.h +++ /dev/null @@ -1,26 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0-only */ -/* - * Copyright (C) 2017 Linaro Ltd. - */ -#ifndef __VENUS_FIRMWARE_H__ -#define __VENUS_FIRMWARE_H__ - -struct device; - -int venus_firmware_init(struct venus_core *core); -void venus_firmware_deinit(struct venus_core *core); -int venus_boot(struct venus_core *core); -int venus_shutdown(struct venus_core *core); -int venus_set_hw_state(struct venus_core *core, bool suspend); - -static inline int venus_set_hw_state_suspend(struct venus_core *core) -{ - return venus_set_hw_state(core, false); -} - -static inline int venus_set_hw_state_resume(struct venus_core *core) -{ - return venus_set_hw_state(core, true); -} - -#endif From patchwork Mon Dec 18 11:31:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755800 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1922F1A714; Mon, 18 Dec 2023 11:33:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="L1ZCkcZ+" Received: from pps.filterd (m0279871.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAscO4000368; Mon, 18 Dec 2023 11:33:09 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=bQdmZd+idGctlHl2svbf9r/7EV7zOUrFdIGTxbDrfGw=; b=L1 ZCkcZ+FADLcg9PK5YrH7PTFfzrjtQFTS0kPCHxnZeZzQEdPVI3Ag0cbfYKL2iN5X za8ydKYd9/c3ap0GkXNr7/YQm9Q92jTL5SXY+909LBhrOtrGRqdfBSJKLMoJm72m /l3JM8lljqOhwPgV6LoHG95065UBmdZGFJJsGRBNcMeJufgq2P8Y6Ner7EG/iKG5 AVfswmbtXcCequIUJPvhamvvCAnW6bt+iFNj6iXosu5w3ZxJFIorMUi0hwsy8785 YoThpU2R8KRXu3lmIfvlK3hbg3jUm8X64SaP9x6hFNqORcC+VJkLmnFWkPJlv6ge CAuufTZVq/eBWo755Lwg== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v14xy443w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:33:09 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5Xe029942; Mon, 18 Dec 2023 11:33:05 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyg6-1; Mon, 18 Dec 2023 11:33:05 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX50h029913; Mon, 18 Dec 2023 11:33:05 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX530029906; Mon, 18 Dec 2023 11:33:05 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 23B6722BC; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 02/34] media: introduce common helpers for queues handling Date: Mon, 18 Dec 2023 17:01:57 +0530 Message-Id: <1702899149-21321-3-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: 4G0iKqfqT_8ZcAehWwbFf1zCDz1xgW1F X-Proofpoint-ORIG-GUID: 4G0iKqfqT_8ZcAehWwbFf1zCDz1xgW1F X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 malwarescore=0 phishscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 impostorscore=0 lowpriorityscore=0 suspectscore=0 spamscore=0 bulkscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Introduce common helpers for shared queues init, deinit and read/write queue operations and use them in venus driver. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/hfi_queue.c | 258 +++++++++++ drivers/media/platform/qcom/vcodec/hfi_queue.h | 129 ++++++ drivers/media/platform/qcom/vcodec/venus/Makefile | 3 +- .../media/platform/qcom/vcodec/venus/hfi_venus.c | 500 ++++----------------- 4 files changed, 465 insertions(+), 425 deletions(-) create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.c create mode 100644 drivers/media/platform/qcom/vcodec/hfi_queue.h diff --git a/drivers/media/platform/qcom/vcodec/hfi_queue.c b/drivers/media/platform/qcom/vcodec/hfi_queue.c new file mode 100644 index 0000000..e3e470b --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/hfi_queue.c @@ -0,0 +1,258 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "hfi_queue.h" + +int write_queue(struct iface_q_info *qinfo, void *packet, + u32 packet_size, u32 *rx_req) +{ + u32 empty_space, read_idx, write_idx, new_write_idx; + struct hfi_queue_header *queue; + u32 *write_ptr, residue; + + queue = qinfo->qhdr; + if (!queue) + return -EINVAL; + + if (!packet_size || packet_size > qinfo->q_array.size) + return -EINVAL; + + read_idx = queue->qhdr_read_idx * sizeof(u32); + write_idx = queue->qhdr_write_idx * sizeof(u32); + + empty_space = (write_idx >= read_idx) ? + (qinfo->q_array.size - (write_idx - read_idx)) : + (read_idx - write_idx); + if (empty_space <= packet_size) { + queue->qhdr_tx_req = 1; + return -ENOSPC; + } + + queue->qhdr_tx_req = 0; + + new_write_idx = write_idx + packet_size; + write_ptr = (u32 *)((u8 *)qinfo->q_array.kernel_vaddr + write_idx); + + if (write_ptr < (u32 *)qinfo->q_array.kernel_vaddr || + write_ptr > (u32 *)(qinfo->q_array.kernel_vaddr + + qinfo->q_array.size)) + return -EINVAL; + + if (new_write_idx < qinfo->q_array.size) { + memcpy(write_ptr, packet, packet_size); + } else { + residue = new_write_idx - qinfo->q_array.size; + memcpy(write_ptr, packet, (packet_size - residue)); + memcpy(qinfo->q_array.kernel_vaddr, + packet + (packet_size - residue), residue); + new_write_idx = residue; + } + + /* Make sure packet is written before updating the write index */ + mb(); + queue->qhdr_write_idx = new_write_idx / sizeof(u32); + *rx_req = queue->qhdr_rx_req ? 1 : 0; + + /* Make sure write index is updated before an interrupt is raised */ + mb(); + + return 0; +} + +int read_queue(struct iface_q_info *qinfo, void *packet, u32 *tx_req) +{ + u32 read_idx, write_idx, new_read_idx; + struct hfi_queue_header *queue; + u32 receive_request = 0; + u32 packet_size, residue; + u32 *read_ptr; + int ret = 0; + + /* Make sure data is valid before reading it */ + mb(); + queue = qinfo->qhdr; + if (!queue) + return -EINVAL; + + if (queue->qhdr_type & IFACEQ_MSGQ_ID) + receive_request = 1; + + read_idx = queue->qhdr_read_idx * sizeof(u32); + write_idx = queue->qhdr_write_idx * sizeof(u32); + + if (read_idx == write_idx) { + queue->qhdr_rx_req = receive_request; + /* Ensure qhdr is updated in main memory */ + mb(); + return -ENODATA; + } + + read_ptr = (u32 *)(qinfo->q_array.kernel_vaddr + read_idx); + if (read_ptr < (u32 *)qinfo->q_array.kernel_vaddr || + read_ptr > (u32 *)(qinfo->q_array.kernel_vaddr + + qinfo->q_array.size - sizeof(*read_ptr))) + return -ENODATA; + + packet_size = *read_ptr; + if (!packet_size) + return -EINVAL; + + new_read_idx = read_idx + packet_size; + if (packet_size <= IFACEQ_CORE_PKT_SIZE && + read_idx <= qinfo->q_array.size) { + if (new_read_idx < qinfo->q_array.size) { + memcpy(packet, read_ptr, packet_size); + } else { + residue = new_read_idx - qinfo->q_array.size; + memcpy(packet, read_ptr, (packet_size - residue)); + memcpy((packet + (packet_size - residue)), + qinfo->q_array.kernel_vaddr, residue); + new_read_idx = residue; + } + } else { + new_read_idx = write_idx; + ret = -EBADMSG; + } + + queue->qhdr_rx_req = receive_request; + + queue->qhdr_read_idx = new_read_idx / sizeof(u32); + + *tx_req = queue->qhdr_tx_req ? 1 : 0; + /* Ensure qhdr is updated in main memory */ + mb(); + + return ret; +} + +static void __set_queue_hdr_defaults(struct hfi_queue_header *q_hdr) +{ + q_hdr->qhdr_status = 0x1; + q_hdr->qhdr_type = IFACEQ_DFLT_QHDR; + q_hdr->qhdr_q_size = IFACEQ_QUEUE_SIZE / 4; + q_hdr->qhdr_pkt_size = 0; + q_hdr->qhdr_rx_wm = 0x1; + q_hdr->qhdr_tx_wm = 0x1; + q_hdr->qhdr_rx_req = 0x1; + q_hdr->qhdr_tx_req = 0x0; + q_hdr->qhdr_rx_irq_status = 0x0; + q_hdr->qhdr_tx_irq_status = 0x0; + q_hdr->qhdr_read_idx = 0x0; + q_hdr->qhdr_write_idx = 0x0; +} + +static void hfi_set_queue_header(u32 queue_id, struct iface_q_info *iface_q) +{ + __set_queue_hdr_defaults(iface_q->qhdr); + iface_q->qhdr->qhdr_start_addr = iface_q->q_array.device_addr; + iface_q->qhdr->qhdr_type |= queue_id; + + /* + * Set receive request to zero on debug queue as there is no + * need of interrupt from video hardware for debug messages + */ + if (queue_id == IFACEQ_DBGQ_ID) + iface_q->qhdr->qhdr_rx_req = 0; +} + +static void __queue_init(struct mem_desc *q_table, u32 queue_id, struct iface_q_info *iface_q) +{ + unsigned int offset = 0; + + offset = q_table->size + (queue_id * IFACEQ_QUEUE_SIZE); + iface_q->q_array.device_addr = q_table->device_addr + offset; + iface_q->q_array.kernel_vaddr = + (void *)((char *)q_table->kernel_vaddr + offset); + iface_q->q_array.size = IFACEQ_QUEUE_SIZE; + iface_q->qhdr = + IFACEQ_GET_QHDR_START_ADDR(q_table->kernel_vaddr, queue_id); + + memset(iface_q->qhdr, 0, sizeof(struct hfi_queue_header)); + + hfi_set_queue_header(queue_id, iface_q); +} + +int hfi_queue_init(struct device *dev, struct mem_desc *q_table, struct mem_desc *sfr, + struct iface_q_info *cmd_q, struct iface_q_info *msg_q, + struct iface_q_info *dbg_q, void *daddr) +{ + struct hfi_queue_table_header *q_tbl_hdr; + + if (q_table->kernel_vaddr) { + hfi_set_queue_header(IFACEQ_CMDQ_ID, cmd_q); + hfi_set_queue_header(IFACEQ_MSGQ_ID, msg_q); + hfi_set_queue_header(IFACEQ_DBGQ_ID, dbg_q); + return 0; + } + + q_table->kernel_vaddr = dma_alloc_attrs(dev, ALIGNED_QUEUE_SIZE, + &q_table->device_addr, + GFP_KERNEL, DMA_ATTR_WRITE_COMBINE); + if (!q_table->kernel_vaddr) { + dev_err(dev, "%s: queues alloc and map failed\n", __func__); + return -ENOMEM; + } + + sfr->kernel_vaddr = dma_alloc_attrs(dev, ALIGNED_SFR_SIZE, + &sfr->device_addr, + GFP_KERNEL, DMA_ATTR_WRITE_COMBINE); + if (!sfr->kernel_vaddr) { + dev_err(dev, "%s: sfr alloc and map failed\n", __func__); + return -ENOMEM; + } + + q_table->size = IFACEQ_TABLE_SIZE; + + __queue_init(q_table, IFACEQ_CMDQ_ID, cmd_q); + __queue_init(q_table, IFACEQ_MSGQ_ID, msg_q); + __queue_init(q_table, IFACEQ_DBGQ_ID, dbg_q); + + q_tbl_hdr = (struct hfi_queue_table_header *)q_table->kernel_vaddr; + q_tbl_hdr->qtbl_version = 0; + q_tbl_hdr->device_addr = (void *)daddr; + strscpy(q_tbl_hdr->name, "hfi-queues", sizeof(q_tbl_hdr->name)); + q_tbl_hdr->qtbl_size = IFACEQ_TABLE_SIZE; + q_tbl_hdr->qtbl_qhdr0_offset = sizeof(*q_tbl_hdr); + q_tbl_hdr->qtbl_qhdr_size = sizeof(struct hfi_queue_header); + q_tbl_hdr->qtbl_num_q = IFACEQ_NUMQ; + q_tbl_hdr->qtbl_num_active_q = IFACEQ_NUMQ; + + sfr->size = ALIGNED_SFR_SIZE; + /* Write sfr size in first word to be used by firmware */ + *((u32 *)sfr->kernel_vaddr) = sfr->size; + + return 0; +} + +static void __queue_deinit(struct iface_q_info *iface_q) +{ + iface_q->qhdr = NULL; + iface_q->q_array.kernel_vaddr = NULL; + iface_q->q_array.device_addr = 0; +} + +void hfi_queue_deinit(struct device *dev, struct mem_desc *q_table, struct mem_desc *sfr, + struct iface_q_info *cmd_q, struct iface_q_info *msg_q, + struct iface_q_info *dbg_q) +{ + if (!q_table->kernel_vaddr) + return; + + dma_free_attrs(dev, q_table->size, q_table->kernel_vaddr, + q_table->device_addr, q_table->attrs); + + dma_free_attrs(dev, sfr->size, sfr->kernel_vaddr, + sfr->device_addr, sfr->attrs); + + __queue_deinit(cmd_q); + __queue_deinit(msg_q); + __queue_deinit(dbg_q); + + q_table->kernel_vaddr = NULL; + q_table->device_addr = 0; + + sfr->kernel_vaddr = NULL; + sfr->device_addr = 0; +} diff --git a/drivers/media/platform/qcom/vcodec/hfi_queue.h b/drivers/media/platform/qcom/vcodec/hfi_queue.h new file mode 100644 index 0000000..8cc9491 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/hfi_queue.h @@ -0,0 +1,129 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _HFI_QUEUE_H_ +#define _HFI_QUEUE_H_ + +#include +#include + +/* + * Shared queues are used for communication between driver and firmware. + * There are 3 types of queues: + * Command queue - driver to write any command to firmware. + * Message queue - firmware to send any response to driver. + * Debug queue - firmware to write debug message. + */ + +/* Host-firmware shared queue ids */ +enum iris_iface_queue { + IFACEQ_CMDQ_ID = 0, + IFACEQ_MSGQ_ID, + IFACEQ_DBGQ_ID, + IFACEQ_NUMQ, /* not an index */ +}; + +#define IFACEQ_MAX_BUF_COUNT 50 +/* + * Max session supported are 16. + * this value is used to calculate the size of + * individual shared queue. + */ +#define IFACE_MAX_PARALLEL_CLNTS 16 +#define IFACEQ_DFLT_QHDR 0x01010000 + +#define IFACEQ_MAX_PKT_SIZE 1024 +#define IFACEQ_CORE_PKT_SIZE (1024 * 4) + +#define IFACEQ_TABLE_SIZE (sizeof(struct hfi_queue_table_header) + \ + sizeof(struct hfi_queue_header) * IFACEQ_NUMQ) + +#define IFACEQ_QUEUE_SIZE (IFACEQ_MAX_PKT_SIZE * \ + IFACEQ_MAX_BUF_COUNT * IFACE_MAX_PARALLEL_CLNTS) + +#define IFACEQ_GET_QHDR_START_ADDR(ptr, i) \ + ((void *)(((ptr) + sizeof(struct hfi_queue_table_header)) + \ + ((i) * sizeof(struct hfi_queue_header)))) + +#define SFR_SIZE SZ_4K +#define QUEUE_SIZE (IFACEQ_TABLE_SIZE + \ + (IFACEQ_QUEUE_SIZE * IFACEQ_NUMQ)) + +#define ALIGNED_SFR_SIZE ALIGN(SFR_SIZE, SZ_4K) +#define ALIGNED_QUEUE_SIZE ALIGN(QUEUE_SIZE, SZ_4K) +#define SHARED_QSIZE ALIGN(ALIGNED_SFR_SIZE + ALIGNED_QUEUE_SIZE, SZ_1M) + +struct mem_desc { + dma_addr_t device_addr; + void *kernel_vaddr; + u32 size; + unsigned long attrs; +}; + +struct hfi_queue_table_header { + u32 qtbl_version; + u32 qtbl_size; + u32 qtbl_qhdr0_offset; + u32 qtbl_qhdr_size; + u32 qtbl_num_q; + u32 qtbl_num_active_q; + void *device_addr; + char name[256]; +}; + +/** + * struct hfi_queue_header - queue header definition + * + * @qhdr_status: bits (7:0) 0x1 - active 0x0 - inactive + * @qhdr_start_addr: queue start addr. + * @qhdr_type: queue type. + * @qhdr_q_size: size in number of HFI_QUEUE_PACKET_TYPE packets it can hold. + * @qhdr_pkt_size: size of the Queue packet entries, in bytes. + * @qhdr_rx_wm: the receiver watermark in number of queue packets. + * @qhdr_tx_wm: the transmitter watermark in number of queue packets. + * @qhdr_rx_req: set to request an interrupt from transmitter if qhdr_tx_wm is reached. + * @qhdr_tx_req: set to request an interrupt from receiver if qhdr_rx_wm is reached. + * @qhdr_rx_irq_status: status of receiver irq. + * @qhdr_tx_irq_status: status of transmitter irq. + * @qhdr_read_idx: read index, which starts with a value of 0 and wraps around after + * (QHDR_Q_SIZE - 1). + * @qhdr_write_idx: write index, which starts with a value of 0 and wraps around after + * (QHDR_Q_SIZE - 1). + */ + +struct hfi_queue_header { + u32 qhdr_status; + u32 qhdr_start_addr; + u32 qhdr_type; + u32 qhdr_q_size; + u32 qhdr_pkt_size; + u32 qhdr_pkt_drop_cnt; + u32 qhdr_rx_wm; + u32 qhdr_tx_wm; + u32 qhdr_rx_req; + u32 qhdr_tx_req; + u32 qhdr_rx_irq_status; + u32 qhdr_tx_irq_status; + u32 qhdr_read_idx; + u32 qhdr_write_idx; +}; + +struct iface_q_info { + struct hfi_queue_header *qhdr; + struct mem_desc q_array; +}; + +int write_queue(struct iface_q_info *qinfo, void *packet, + u32 packet_size, u32 *rx_req); +int read_queue(struct iface_q_info *qinfo, void *packet, u32 *tx_req); +int hfi_queue_init(struct device *dev, struct mem_desc *q_table, struct mem_desc *sfr, + struct iface_q_info *cmd_q, struct iface_q_info *msg_q, + struct iface_q_info *dbg_q, void *daddr); + +void hfi_queue_deinit(struct device *dev, struct mem_desc *q_table, struct mem_desc *sfr, + struct iface_q_info *cmd_q, struct iface_q_info *msg_q, + struct iface_q_info *dbg_q); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/venus/Makefile b/drivers/media/platform/qcom/vcodec/venus/Makefile index f6f3a88..1941ef4 100644 --- a/drivers/media/platform/qcom/vcodec/venus/Makefile +++ b/drivers/media/platform/qcom/vcodec/venus/Makefile @@ -1,7 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 # Makefile for Qualcomm Venus driver -venus-core-objs += ../firmware.o +venus-core-objs += ../firmware.o \ + ../hfi_queue.o venus-core-objs += core.o helpers.o firmware_no_tz.o \ hfi_venus.o hfi_msgs.o hfi_cmds.o hfi.o \ diff --git a/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c index 5a68db9..41e3f27 100644 --- a/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c +++ b/drivers/media/platform/qcom/vcodec/venus/hfi_venus.c @@ -16,6 +16,7 @@ #include "../firmware.h" #include "hfi_cmds.h" #include "hfi_msgs.h" +#include "../hfi_queue.h" #include "hfi_venus.h" #include "hfi_venus_io.h" #include "firmware_no_tz.h" @@ -34,79 +35,13 @@ #define IFACEQ_CMD_IDX 0 #define IFACEQ_MSG_IDX 1 #define IFACEQ_DBG_IDX 2 -#define IFACEQ_MAX_BUF_COUNT 50 -#define IFACEQ_MAX_PARALLEL_CLNTS 16 -#define IFACEQ_DFLT_QHDR 0x01010000 #define POLL_INTERVAL_US 50 -#define IFACEQ_MAX_PKT_SIZE 1024 -#define IFACEQ_MED_PKT_SIZE 768 -#define IFACEQ_MIN_PKT_SIZE 8 #define IFACEQ_VAR_SMALL_PKT_SIZE 100 #define IFACEQ_VAR_LARGE_PKT_SIZE 512 #define IFACEQ_VAR_HUGE_PKT_SIZE (1024 * 12) -struct hfi_queue_table_header { - u32 version; - u32 size; - u32 qhdr0_offset; - u32 qhdr_size; - u32 num_q; - u32 num_active_q; -}; - -struct hfi_queue_header { - u32 status; - u32 start_addr; - u32 type; - u32 q_size; - u32 pkt_size; - u32 pkt_drop_cnt; - u32 rx_wm; - u32 tx_wm; - u32 rx_req; - u32 tx_req; - u32 rx_irq_status; - u32 tx_irq_status; - u32 read_idx; - u32 write_idx; -}; - -#define IFACEQ_TABLE_SIZE \ - (sizeof(struct hfi_queue_table_header) + \ - sizeof(struct hfi_queue_header) * IFACEQ_NUM) - -#define IFACEQ_QUEUE_SIZE (IFACEQ_MAX_PKT_SIZE * \ - IFACEQ_MAX_BUF_COUNT * IFACEQ_MAX_PARALLEL_CLNTS) - -#define IFACEQ_GET_QHDR_START_ADDR(ptr, i) \ - (void *)(((ptr) + sizeof(struct hfi_queue_table_header)) + \ - ((i) * sizeof(struct hfi_queue_header))) - -#define QDSS_SIZE SZ_4K -#define SFR_SIZE SZ_4K -#define QUEUE_SIZE \ - (IFACEQ_TABLE_SIZE + (IFACEQ_QUEUE_SIZE * IFACEQ_NUM)) - -#define ALIGNED_QDSS_SIZE ALIGN(QDSS_SIZE, SZ_4K) -#define ALIGNED_SFR_SIZE ALIGN(SFR_SIZE, SZ_4K) -#define ALIGNED_QUEUE_SIZE ALIGN(QUEUE_SIZE, SZ_4K) -#define SHARED_QSIZE ALIGN(ALIGNED_SFR_SIZE + ALIGNED_QUEUE_SIZE + \ - ALIGNED_QDSS_SIZE, SZ_1M) - -struct mem_desc { - dma_addr_t da; /* device address */ - void *kva; /* kernel virtual address */ - u32 size; - unsigned long attrs; -}; - -struct iface_queue { - struct hfi_queue_header *qhdr; - struct mem_desc qmem; -}; - enum venus_state { VENUS_STATE_DEINIT = 1, VENUS_STATE_INIT, @@ -125,12 +60,11 @@ struct venus_hfi_device { struct completion release_resource; struct mem_desc ifaceq_table; struct mem_desc sfr; - struct iface_queue queues[IFACEQ_NUM]; + struct iface_q_info queues[IFACEQ_NUM]; u8 pkt_buf[IFACEQ_VAR_HUGE_PKT_SIZE]; u8 dbg_buf[IFACEQ_VAR_HUGE_PKT_SIZE]; }; -static bool venus_pkt_debug; int venus_fw_debug = HFI_DEBUG_MSG_ERROR | HFI_DEBUG_MSG_FATAL; static bool venus_fw_low_power_mode = true; static int venus_hw_rsp_timeout = 1000; @@ -149,212 +83,6 @@ static bool venus_is_valid_state(struct venus_hfi_device *hdev) return hdev->state != VENUS_STATE_DEINIT; } -static void venus_dump_packet(struct venus_hfi_device *hdev, const void *packet) -{ - size_t pkt_size = *(u32 *)packet; - - if (!venus_pkt_debug) - return; - - print_hex_dump(KERN_DEBUG, "", DUMP_PREFIX_OFFSET, 16, 1, packet, - pkt_size, true); -} - -static int venus_write_queue(struct venus_hfi_device *hdev, - struct iface_queue *queue, - void *packet, u32 *rx_req) -{ - struct hfi_queue_header *qhdr; - u32 dwords, new_wr_idx; - u32 empty_space, rd_idx, wr_idx, qsize; - u32 *wr_ptr; - - if (!queue->qmem.kva) - return -EINVAL; - - qhdr = queue->qhdr; - if (!qhdr) - return -EINVAL; - - venus_dump_packet(hdev, packet); - - dwords = (*(u32 *)packet) >> 2; - if (!dwords) - return -EINVAL; - - rd_idx = qhdr->read_idx; - wr_idx = qhdr->write_idx; - qsize = qhdr->q_size; - /* ensure rd/wr indices's are read from memory */ - rmb(); - - if (wr_idx >= rd_idx) - empty_space = qsize - (wr_idx - rd_idx); - else - empty_space = rd_idx - wr_idx; - - if (empty_space <= dwords) { - qhdr->tx_req = 1; - /* ensure tx_req is updated in memory */ - wmb(); - return -ENOSPC; - } - - qhdr->tx_req = 0; - /* ensure tx_req is updated in memory */ - wmb(); - - new_wr_idx = wr_idx + dwords; - wr_ptr = (u32 *)(queue->qmem.kva + (wr_idx << 2)); - - if (wr_ptr < (u32 *)queue->qmem.kva || - wr_ptr > (u32 *)(queue->qmem.kva + queue->qmem.size - sizeof(*wr_ptr))) - return -EINVAL; - - if (new_wr_idx < qsize) { - memcpy(wr_ptr, packet, dwords << 2); - } else { - size_t len; - - new_wr_idx -= qsize; - len = (dwords - new_wr_idx) << 2; - memcpy(wr_ptr, packet, len); - memcpy(queue->qmem.kva, packet + len, new_wr_idx << 2); - } - - /* make sure packet is written before updating the write index */ - wmb(); - - qhdr->write_idx = new_wr_idx; - *rx_req = qhdr->rx_req ? 1 : 0; - - /* make sure write index is updated before an interrupt is raised */ - mb(); - - return 0; -} - -static int venus_read_queue(struct venus_hfi_device *hdev, - struct iface_queue *queue, void *pkt, u32 *tx_req) -{ - struct hfi_queue_header *qhdr; - u32 dwords, new_rd_idx; - u32 rd_idx, wr_idx, type, qsize; - u32 *rd_ptr; - u32 recv_request = 0; - int ret = 0; - - if (!queue->qmem.kva) - return -EINVAL; - - qhdr = queue->qhdr; - if (!qhdr) - return -EINVAL; - - type = qhdr->type; - rd_idx = qhdr->read_idx; - wr_idx = qhdr->write_idx; - qsize = qhdr->q_size; - - /* make sure data is valid before using it */ - rmb(); - - /* - * Do not set receive request for debug queue, if set, Venus generates - * interrupt for debug messages even when there is no response message - * available. In general debug queue will not become full as it is being - * emptied out for every interrupt from Venus. Venus will anyway - * generates interrupt if it is full. - */ - if (type & HFI_CTRL_TO_HOST_MSG_Q) - recv_request = 1; - - if (rd_idx == wr_idx) { - qhdr->rx_req = recv_request; - *tx_req = 0; - /* update rx_req field in memory */ - wmb(); - return -ENODATA; - } - - rd_ptr = (u32 *)(queue->qmem.kva + (rd_idx << 2)); - - if (rd_ptr < (u32 *)queue->qmem.kva || - rd_ptr > (u32 *)(queue->qmem.kva + queue->qmem.size - sizeof(*rd_ptr))) - return -EINVAL; - - dwords = *rd_ptr >> 2; - if (!dwords) - return -EINVAL; - - new_rd_idx = rd_idx + dwords; - if (((dwords << 2) <= IFACEQ_VAR_HUGE_PKT_SIZE) && rd_idx <= qsize) { - if (new_rd_idx < qsize) { - memcpy(pkt, rd_ptr, dwords << 2); - } else { - size_t len; - - new_rd_idx -= qsize; - len = (dwords - new_rd_idx) << 2; - memcpy(pkt, rd_ptr, len); - memcpy(pkt + len, queue->qmem.kva, new_rd_idx << 2); - } - } else { - /* bad packet received, dropping */ - new_rd_idx = qhdr->write_idx; - ret = -EBADMSG; - } - - /* ensure the packet is read before updating read index */ - rmb(); - - qhdr->read_idx = new_rd_idx; - /* ensure updating read index */ - wmb(); - - rd_idx = qhdr->read_idx; - wr_idx = qhdr->write_idx; - /* ensure rd/wr indices are read from memory */ - rmb(); - - if (rd_idx != wr_idx) - qhdr->rx_req = 0; - else - qhdr->rx_req = recv_request; - - *tx_req = qhdr->tx_req ? 1 : 0; - - /* ensure rx_req is stored to memory and tx_req is loaded from memory */ - mb(); - - venus_dump_packet(hdev, pkt); - - return ret; -} - -static int venus_alloc(struct venus_hfi_device *hdev, struct mem_desc *desc, - u32 size) -{ - struct device *dev = hdev->core->dev; - - desc->attrs = DMA_ATTR_WRITE_COMBINE; - desc->size = ALIGN(size, SZ_4K); - - desc->kva = dma_alloc_attrs(dev, desc->size, &desc->da, GFP_KERNEL, - desc->attrs); - if (!desc->kva) - return -ENOMEM; - - return 0; -} - -static void venus_free(struct venus_hfi_device *hdev, struct mem_desc *mem) -{ - struct device *dev = hdev->core->dev; - - dma_free_attrs(dev, mem->size, mem->kva, mem->da, mem->attrs); -} - static void venus_set_registers(struct venus_hfi_device *hdev) { const struct venus_resources *res = hdev->core->res; @@ -380,11 +108,11 @@ static void venus_soft_int(struct venus_hfi_device *hdev) } static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev, - void *pkt, bool sync) + void *pkt, u32 pkt_size, bool sync) { struct device *dev = hdev->core->dev; struct hfi_pkt_hdr *cmd_packet; - struct iface_queue *queue; + struct iface_q_info *queue; u32 rx_req; int ret; @@ -396,7 +124,7 @@ static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev, queue = &hdev->queues[IFACEQ_CMD_IDX]; - ret = venus_write_queue(hdev, queue, pkt, &rx_req); + ret = write_queue(queue, pkt, pkt_size, &rx_req); if (ret) { dev_err(dev, "write to iface cmd queue failed (%d)\n", ret); return ret; @@ -408,7 +136,7 @@ static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev, * commands */ queue = &hdev->queues[IFACEQ_MSG_IDX]; - queue->qhdr->rx_req = 1; + queue->qhdr->qhdr_rx_req = 1; /* ensure rx_req is updated in memory */ wmb(); } @@ -419,12 +147,12 @@ static int venus_iface_cmdq_write_nolock(struct venus_hfi_device *hdev, return 0; } -static int venus_iface_cmdq_write(struct venus_hfi_device *hdev, void *pkt, bool sync) +static int venus_iface_cmdq_write(struct venus_hfi_device *hdev, void *pkt, u32 pkt_size, bool sync) { int ret; mutex_lock(&hdev->lock); - ret = venus_iface_cmdq_write_nolock(hdev, pkt, sync); + ret = venus_iface_cmdq_write_nolock(hdev, pkt, pkt_size, sync); mutex_unlock(&hdev->lock); return ret; @@ -447,7 +175,7 @@ static int venus_hfi_core_set_resource(struct venus_core *core, u32 id, if (ret) return ret; - ret = venus_iface_cmdq_write(hdev, pkt, false); + ret = venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false); if (ret) return ret; @@ -531,12 +259,12 @@ static int venus_run(struct venus_hfi_device *hdev) */ venus_set_registers(hdev); - writel(hdev->ifaceq_table.da, cpu_cs_base + UC_REGION_ADDR); + writel(hdev->ifaceq_table.device_addr, cpu_cs_base + UC_REGION_ADDR); writel(SHARED_QSIZE, cpu_cs_base + UC_REGION_SIZE); - writel(hdev->ifaceq_table.da, cpu_cs_base + CPU_CS_SCIACMDARG2); + writel(hdev->ifaceq_table.device_addr, cpu_cs_base + CPU_CS_SCIACMDARG2); writel(0x01, cpu_cs_base + CPU_CS_SCIACMDARG1); - if (hdev->sfr.da) - writel(hdev->sfr.da, cpu_cs_base + SFR_ADDR); + if (hdev->sfr.device_addr) + writel(hdev->sfr.device_addr, cpu_cs_base + SFR_ADDR); ret = venus_boot_core(hdev); if (ret) { @@ -690,7 +418,7 @@ static int venus_power_on(struct venus_hfi_device *hdev) static int venus_iface_msgq_read_nolock(struct venus_hfi_device *hdev, void *pkt) { - struct iface_queue *queue; + struct iface_q_info *queue; u32 tx_req; int ret; @@ -699,7 +427,7 @@ static int venus_iface_msgq_read_nolock(struct venus_hfi_device *hdev, queue = &hdev->queues[IFACEQ_MSG_IDX]; - ret = venus_read_queue(hdev, queue, pkt, &tx_req); + ret = read_queue(queue, pkt, &tx_req); if (ret) return ret; @@ -723,7 +451,7 @@ static int venus_iface_msgq_read(struct venus_hfi_device *hdev, void *pkt) static int venus_iface_dbgq_read_nolock(struct venus_hfi_device *hdev, void *pkt) { - struct iface_queue *queue; + struct iface_q_info *queue; u32 tx_req; int ret; @@ -733,7 +461,7 @@ static int venus_iface_dbgq_read_nolock(struct venus_hfi_device *hdev, queue = &hdev->queues[IFACEQ_DBG_IDX]; - ret = venus_read_queue(hdev, queue, pkt, &tx_req); + ret = read_queue(queue, pkt, &tx_req); if (ret) return ret; @@ -759,100 +487,18 @@ static int venus_iface_dbgq_read(struct venus_hfi_device *hdev, void *pkt) static void venus_set_qhdr_defaults(struct hfi_queue_header *qhdr) { - qhdr->status = 1; - qhdr->type = IFACEQ_DFLT_QHDR; - qhdr->q_size = IFACEQ_QUEUE_SIZE / 4; - qhdr->pkt_size = 0; - qhdr->rx_wm = 1; - qhdr->tx_wm = 1; - qhdr->rx_req = 1; - qhdr->tx_req = 0; - qhdr->rx_irq_status = 0; - qhdr->tx_irq_status = 0; - qhdr->read_idx = 0; - qhdr->write_idx = 0; -} - -static void venus_interface_queues_release(struct venus_hfi_device *hdev) -{ - mutex_lock(&hdev->lock); - - venus_free(hdev, &hdev->ifaceq_table); - venus_free(hdev, &hdev->sfr); - - memset(hdev->queues, 0, sizeof(hdev->queues)); - memset(&hdev->ifaceq_table, 0, sizeof(hdev->ifaceq_table)); - memset(&hdev->sfr, 0, sizeof(hdev->sfr)); - - mutex_unlock(&hdev->lock); -} - -static int venus_interface_queues_init(struct venus_hfi_device *hdev) -{ - struct hfi_queue_table_header *tbl_hdr; - struct iface_queue *queue; - struct hfi_sfr *sfr; - struct mem_desc desc = {0}; - unsigned int offset; - unsigned int i; - int ret; - - ret = venus_alloc(hdev, &desc, ALIGNED_QUEUE_SIZE); - if (ret) - return ret; - - hdev->ifaceq_table = desc; - offset = IFACEQ_TABLE_SIZE; - - for (i = 0; i < IFACEQ_NUM; i++) { - queue = &hdev->queues[i]; - queue->qmem.da = desc.da + offset; - queue->qmem.kva = desc.kva + offset; - queue->qmem.size = IFACEQ_QUEUE_SIZE; - offset += queue->qmem.size; - queue->qhdr = - IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kva, i); - - venus_set_qhdr_defaults(queue->qhdr); - - queue->qhdr->start_addr = queue->qmem.da; - - if (i == IFACEQ_CMD_IDX) - queue->qhdr->type |= HFI_HOST_TO_CTRL_CMD_Q; - else if (i == IFACEQ_MSG_IDX) - queue->qhdr->type |= HFI_CTRL_TO_HOST_MSG_Q; - else if (i == IFACEQ_DBG_IDX) - queue->qhdr->type |= HFI_CTRL_TO_HOST_DBG_Q; - } - - tbl_hdr = hdev->ifaceq_table.kva; - tbl_hdr->version = 0; - tbl_hdr->size = IFACEQ_TABLE_SIZE; - tbl_hdr->qhdr0_offset = sizeof(struct hfi_queue_table_header); - tbl_hdr->qhdr_size = sizeof(struct hfi_queue_header); - tbl_hdr->num_q = IFACEQ_NUM; - tbl_hdr->num_active_q = IFACEQ_NUM; - - /* - * Set receive request to zero on debug queue as there is no - * need of interrupt from video hardware for debug messages - */ - queue = &hdev->queues[IFACEQ_DBG_IDX]; - queue->qhdr->rx_req = 0; - - ret = venus_alloc(hdev, &desc, ALIGNED_SFR_SIZE); - if (ret) { - hdev->sfr.da = 0; - } else { - hdev->sfr = desc; - sfr = hdev->sfr.kva; - sfr->buf_size = ALIGNED_SFR_SIZE; - } - - /* ensure table and queue header structs are settled in memory */ - wmb(); - - return 0; + qhdr->qhdr_status = 1; + qhdr->qhdr_type = IFACEQ_DFLT_QHDR; + qhdr->qhdr_q_size = IFACEQ_QUEUE_SIZE / 4; + qhdr->qhdr_pkt_size = 0; + qhdr->qhdr_rx_wm = 1; + qhdr->qhdr_tx_wm = 1; + qhdr->qhdr_rx_req = 1; + qhdr->qhdr_tx_req = 0; + qhdr->qhdr_rx_irq_status = 0; + qhdr->qhdr_tx_irq_status = 0; + qhdr->qhdr_read_idx = 0; + qhdr->qhdr_write_idx = 0; } static int venus_sys_set_debug(struct venus_hfi_device *hdev, u32 debug) @@ -864,7 +510,7 @@ static int venus_sys_set_debug(struct venus_hfi_device *hdev, u32 debug) pkt_sys_debug_config(pkt, HFI_DEBUG_MODE_QUEUE, debug); - return venus_iface_cmdq_write(hdev, pkt, false); + return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false); } static int venus_sys_set_coverage(struct venus_hfi_device *hdev, u32 mode) @@ -876,7 +522,7 @@ static int venus_sys_set_coverage(struct venus_hfi_device *hdev, u32 mode) pkt_sys_coverage_config(pkt, mode); - return venus_iface_cmdq_write(hdev, pkt, false); + return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false); } static int venus_sys_set_idle_message(struct venus_hfi_device *hdev, @@ -892,7 +538,7 @@ static int venus_sys_set_idle_message(struct venus_hfi_device *hdev, pkt_sys_idle_indicator(pkt, enable); - return venus_iface_cmdq_write(hdev, pkt, false); + return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false); } static int venus_sys_set_power_control(struct venus_hfi_device *hdev, @@ -905,7 +551,7 @@ static int venus_sys_set_power_control(struct venus_hfi_device *hdev, pkt_sys_power_control(pkt, enable); - return venus_iface_cmdq_write(hdev, pkt, false); + return venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false); } static int venus_sys_set_ubwc_config(struct venus_hfi_device *hdev) @@ -919,7 +565,7 @@ static int venus_sys_set_ubwc_config(struct venus_hfi_device *hdev) pkt_sys_ubwc_config(pkt, res->ubwc_conf); - ret = venus_iface_cmdq_write(hdev, pkt, false); + ret = venus_iface_cmdq_write(hdev, pkt, pkt->hdr.size, false); if (ret) return ret; @@ -938,7 +584,7 @@ static int venus_get_queue_size(struct venus_hfi_device *hdev, if (!qhdr) return -EINVAL; - return abs(qhdr->read_idx - qhdr->write_idx); + return abs(qhdr->qhdr_read_idx - qhdr->qhdr_write_idx); } static int venus_sys_set_default_properties(struct venus_hfi_device *hdev) @@ -980,7 +626,7 @@ static int venus_session_cmd(struct venus_inst *inst, u32 pkt_type, bool sync) pkt_session_cmd(&pkt, pkt_type, inst); - return venus_iface_cmdq_write(hdev, &pkt, sync); + return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, sync); } static void venus_flush_debug_queue(struct venus_hfi_device *hdev) @@ -1010,7 +656,7 @@ static int venus_prepare_power_collapse(struct venus_hfi_device *hdev, pkt_sys_pc_prep(&pkt); - ret = venus_iface_cmdq_write(hdev, &pkt, false); + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false); if (ret) return ret; @@ -1047,7 +693,7 @@ static int venus_are_queues_empty(struct venus_hfi_device *hdev) static void venus_sfr_print(struct venus_hfi_device *hdev) { struct device *dev = hdev->core->dev; - struct hfi_sfr *sfr = hdev->sfr.kva; + struct hfi_sfr *sfr = hdev->sfr.kernel_vaddr; void *p; if (!sfr) @@ -1163,13 +809,13 @@ static int venus_core_init(struct venus_core *core) venus_set_state(hdev, VENUS_STATE_INIT); - ret = venus_iface_cmdq_write(hdev, &pkt, false); + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false); if (ret) return ret; pkt_sys_image_version(&version_pkt); - ret = venus_iface_cmdq_write(hdev, &version_pkt, false); + ret = venus_iface_cmdq_write(hdev, &version_pkt, version_pkt.hdr.size, false); if (ret) dev_warn(dev, "failed to send image version pkt to fw\n"); @@ -1198,7 +844,7 @@ static int venus_core_ping(struct venus_core *core, u32 cookie) pkt_sys_ping(&pkt, cookie); - return venus_iface_cmdq_write(hdev, &pkt, false); + return venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false); } static int venus_core_trigger_ssr(struct venus_core *core, u32 trigger_type) @@ -1211,7 +857,7 @@ static int venus_core_trigger_ssr(struct venus_core *core, u32 trigger_type) if (ret) return ret; - return venus_iface_cmdq_write(hdev, &pkt, false); + return venus_iface_cmdq_write(hdev, &pkt, pkt.hdr.size, false); } static int venus_session_init(struct venus_inst *inst, u32 session_type, @@ -1229,7 +875,7 @@ static int venus_session_init(struct venus_inst *inst, u32 session_type, if (ret) goto err; - ret = venus_iface_cmdq_write(hdev, &pkt, true); + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, true); if (ret) goto err; @@ -1272,7 +918,7 @@ static int venus_session_flush(struct venus_inst *inst, u32 flush_mode) if (ret) return ret; - return venus_iface_cmdq_write(hdev, &pkt, true); + return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, true); } static int venus_session_start(struct venus_inst *inst) @@ -1304,7 +950,7 @@ static int venus_session_etb(struct venus_inst *inst, if (ret) return ret; - ret = venus_iface_cmdq_write(hdev, &pkt, false); + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, false); } else if (session_type == VIDC_SESSION_TYPE_ENC) { struct hfi_session_empty_buffer_uncompressed_plane0_pkt pkt; @@ -1312,7 +958,7 @@ static int venus_session_etb(struct venus_inst *inst, if (ret) return ret; - ret = venus_iface_cmdq_write(hdev, &pkt, false); + ret = venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, false); } else { ret = -EINVAL; } @@ -1331,7 +977,7 @@ static int venus_session_ftb(struct venus_inst *inst, if (ret) return ret; - return venus_iface_cmdq_write(hdev, &pkt, false); + return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, false); } static int venus_session_set_buffers(struct venus_inst *inst, @@ -1351,7 +997,7 @@ static int venus_session_set_buffers(struct venus_inst *inst, if (ret) return ret; - return venus_iface_cmdq_write(hdev, pkt, false); + return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false); } static int venus_session_unset_buffers(struct venus_inst *inst, @@ -1371,7 +1017,7 @@ static int venus_session_unset_buffers(struct venus_inst *inst, if (ret) return ret; - return venus_iface_cmdq_write(hdev, pkt, true); + return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, true); } static int venus_session_load_res(struct venus_inst *inst) @@ -1398,7 +1044,7 @@ static int venus_session_parse_seq_hdr(struct venus_inst *inst, u32 seq_hdr, if (ret) return ret; - ret = venus_iface_cmdq_write(hdev, pkt, false); + ret = venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false); if (ret) return ret; @@ -1419,7 +1065,7 @@ static int venus_session_get_seq_hdr(struct venus_inst *inst, u32 seq_hdr, if (ret) return ret; - return venus_iface_cmdq_write(hdev, pkt, false); + return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false); } static int venus_session_set_property(struct venus_inst *inst, u32 ptype, @@ -1438,7 +1084,7 @@ static int venus_session_set_property(struct venus_inst *inst, u32 ptype, if (ret) return ret; - return venus_iface_cmdq_write(hdev, pkt, false); + return venus_iface_cmdq_write(hdev, pkt, pkt->shdr.hdr.size, false); } static int venus_session_get_property(struct venus_inst *inst, u32 ptype) @@ -1451,7 +1097,7 @@ static int venus_session_get_property(struct venus_inst *inst, u32 ptype) if (ret) return ret; - return venus_iface_cmdq_write(hdev, &pkt, true); + return venus_iface_cmdq_write(hdev, &pkt, pkt.shdr.hdr.size, true); } static int venus_resume(struct venus_core *core) @@ -1685,7 +1331,11 @@ void venus_hfi_destroy(struct venus_core *core) struct venus_hfi_device *hdev = to_hfi_priv(core); core->priv = NULL; - venus_interface_queues_release(hdev); + mutex_lock(&hdev->lock); + hfi_queue_deinit(hdev->core->dev, &hdev->ifaceq_table, &hdev->sfr, + &hdev->queues[0], &hdev->queues[1], + &hdev->queues[2]); + mutex_unlock(&hdev->lock); mutex_destroy(&hdev->lock); kfree(hdev); core->ops = NULL; @@ -1707,7 +1357,9 @@ int venus_hfi_create(struct venus_core *core) core->priv = hdev; core->ops = &venus_hfi_ops; - ret = venus_interface_queues_init(hdev); + ret = hfi_queue_init(hdev->core->dev, &hdev->ifaceq_table, &hdev->sfr, + &hdev->queues[0], &hdev->queues[1], + &hdev->queues[2], &hdev->core); if (ret) goto err_kfree; @@ -1724,7 +1376,7 @@ void venus_hfi_queues_reinit(struct venus_core *core) { struct venus_hfi_device *hdev = to_hfi_priv(core); struct hfi_queue_table_header *tbl_hdr; - struct iface_queue *queue; + struct iface_q_info *queue; struct hfi_sfr *sfr; unsigned int i; @@ -1733,36 +1385,36 @@ void venus_hfi_queues_reinit(struct venus_core *core) for (i = 0; i < IFACEQ_NUM; i++) { queue = &hdev->queues[i]; queue->qhdr = - IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kva, i); + IFACEQ_GET_QHDR_START_ADDR(hdev->ifaceq_table.kernel_vaddr, i); venus_set_qhdr_defaults(queue->qhdr); - queue->qhdr->start_addr = queue->qmem.da; + queue->qhdr->qhdr_start_addr = queue->q_array.device_addr; if (i == IFACEQ_CMD_IDX) - queue->qhdr->type |= HFI_HOST_TO_CTRL_CMD_Q; + queue->qhdr->qhdr_type |= HFI_HOST_TO_CTRL_CMD_Q; else if (i == IFACEQ_MSG_IDX) - queue->qhdr->type |= HFI_CTRL_TO_HOST_MSG_Q; + queue->qhdr->qhdr_type |= HFI_CTRL_TO_HOST_MSG_Q; else if (i == IFACEQ_DBG_IDX) - queue->qhdr->type |= HFI_CTRL_TO_HOST_DBG_Q; + queue->qhdr->qhdr_type |= HFI_CTRL_TO_HOST_DBG_Q; } - tbl_hdr = hdev->ifaceq_table.kva; - tbl_hdr->version = 0; - tbl_hdr->size = IFACEQ_TABLE_SIZE; - tbl_hdr->qhdr0_offset = sizeof(struct hfi_queue_table_header); - tbl_hdr->qhdr_size = sizeof(struct hfi_queue_header); - tbl_hdr->num_q = IFACEQ_NUM; - tbl_hdr->num_active_q = IFACEQ_NUM; + tbl_hdr = hdev->ifaceq_table.kernel_vaddr; + tbl_hdr->qtbl_version = 0; + tbl_hdr->qtbl_size = IFACEQ_TABLE_SIZE; + tbl_hdr->qtbl_qhdr0_offset = sizeof(struct hfi_queue_table_header); + tbl_hdr->qtbl_qhdr_size = sizeof(struct hfi_queue_header); + tbl_hdr->qtbl_num_q = IFACEQ_NUM; + tbl_hdr->qtbl_num_active_q = IFACEQ_NUM; /* * Set receive request to zero on debug queue as there is no * need of interrupt from video hardware for debug messages */ queue = &hdev->queues[IFACEQ_DBG_IDX]; - queue->qhdr->rx_req = 0; + queue->qhdr->qhdr_rx_req = 0; - sfr = hdev->sfr.kva; + sfr = hdev->sfr.kernel_vaddr; sfr->buf_size = ALIGNED_SFR_SIZE; /* ensure table and queue header structs are settled in memory */ From patchwork Mon Dec 18 11:32:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755795 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4AE1A199DB; Mon, 18 Dec 2023 11:36:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="ZFmnun9K" Received: from pps.filterd (m0279871.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAsCGw032638; Mon, 18 Dec 2023 11:36:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=LX6X6WF6b2DnvVb0x9SqnHVvPQHNfSxl8FB6K8HP6sw=; b=ZF mnun9KydsAw/asYajUI0JUh42vUrjSf+0MHeYm6RTwMPXdlHC4x7gaVHerzb1KP4 fPSnnQ5cYhW3siUfRYddUSOwSP7IarEXLy4wsgqKKGKpxfuzt2j9JZdyRHnJjV2C qy1ZRRNWIRcwp6i7MFf9vL83N8fCyFhA4RATSx1hYTuNH4k1Oa42dkveDKsMwWzO zU0dHaNWvvYv6NYSQyW/xk0dWaKumFmo375ZtyfV7QLfZ0Uzq/+Wv8QUMNDNfUlx Q+Dr6aBVETfgFzEhbOdZjfyB7ktxnJkh+0oALHkWZaMVH77IA8CtpqcMKczhklt5 ngjdByKMM1+jYQDX89gQ== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v14xy44cs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:36:10 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBVmpM028250; Mon, 18 Dec 2023 11:36:07 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyhc-1; Mon, 18 Dec 2023 11:33:07 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7XZ029998; Mon, 18 Dec 2023 11:33:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX62X029968; Mon, 18 Dec 2023 11:33:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 2E22822C6; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 06/34] media: iris: register video device to platform driver Date: Mon, 18 Dec 2023 17:02:01 +0530 Message-Id: <1702899149-21321-7-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: aYFzfkEloCRmcSdpA9dm31aQYrFM9uqY X-Proofpoint-ORIG-GUID: aYFzfkEloCRmcSdpA9dm31aQYrFM9uqY X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 malwarescore=0 phishscore=0 priorityscore=1501 mlxscore=0 mlxlogscore=999 impostorscore=0 lowpriorityscore=0 suspectscore=0 spamscore=0 bulkscore=0 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Iris is a multi pipe based video acceleration hardware block that offloads video stream encoding and decoding from the application processor (AP). It supports H.264 and H.265 encoding and decoding, as well as VP9 decoding. The AP communicates with hardware through a well defined protocol which provides fine-grained and asynchronous control over individual hardware features. This patch introduces basic probe and remove functions. It handles setting up a video device as well as registering it with the V4L2 subsystem. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/Kconfig | 1 + drivers/media/platform/qcom/Makefile | 1 + drivers/media/platform/qcom/vcodec/iris/Kconfig | 13 +++ drivers/media/platform/qcom/vcodec/iris/Makefile | 3 + .../media/platform/qcom/vcodec/iris/iris_core.h | 34 +++++++ .../media/platform/qcom/vcodec/iris/iris_probe.c | 111 +++++++++++++++++++++ 6 files changed, 163 insertions(+) create mode 100644 drivers/media/platform/qcom/vcodec/iris/Kconfig create mode 100644 drivers/media/platform/qcom/vcodec/iris/Makefile create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_probe.c diff --git a/drivers/media/platform/qcom/Kconfig b/drivers/media/platform/qcom/Kconfig index e94142f..7c88837 100644 --- a/drivers/media/platform/qcom/Kconfig +++ b/drivers/media/platform/qcom/Kconfig @@ -4,3 +4,4 @@ comment "Qualcomm media platform drivers" source "drivers/media/platform/qcom/camss/Kconfig" source "drivers/media/platform/qcom/vcodec/venus/Kconfig" +source "drivers/media/platform/qcom/vcodec/iris/Kconfig" diff --git a/drivers/media/platform/qcom/Makefile b/drivers/media/platform/qcom/Makefile index 3d2d82b..3c76969 100644 --- a/drivers/media/platform/qcom/Makefile +++ b/drivers/media/platform/qcom/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only obj-y += camss/ obj-y += vcodec/venus/ +obj-y += vcodec/iris/ diff --git a/drivers/media/platform/qcom/vcodec/iris/Kconfig b/drivers/media/platform/qcom/vcodec/iris/Kconfig new file mode 100644 index 0000000..850a5b4 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/Kconfig @@ -0,0 +1,13 @@ +config VIDEO_QCOM_IRIS + tristate "Qualcomm Iris V4L2 encoder/decoder driver" + depends on VIDEO_DEV + depends on ARCH_QCOM + select QCOM_MDT_LOADER if ARCH_QCOM + select QCOM_SCM + select DMABUF_HEAPS + select DMABUF_HEAPS_SYSTEM + help + This is a V4L2 driver for Qualcomm Iris video accelerator + hardware. It accelerates encoding and decoding operations + on various Qualcomm SoCs. + To compile this driver as a module choose m here. diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile new file mode 100644 index 0000000..5536ae0 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -0,0 +1,3 @@ +iris-objs += iris_probe.o + +obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h new file mode 100644 index 0000000..ab7fcee --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -0,0 +1,34 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_CORE_H_ +#define _IRIS_CORE_H_ + +#include +#include + +/** + * struct iris_core - holds core parameters valid for all instances + * + * @dev: reference to device structure + * @reg_base: IO memory base address + * @irq: iris irq + * @v4l2_dev: a holder for v4l2 device structure + * @vdev_dec: iris video device structure for decoder + * @v4l2_file_ops: iris v4l2 file ops + * @v4l2_ioctl_ops: iris v4l2 ioctl ops + */ + +struct iris_core { + struct device *dev; + void __iomem *reg_base; + int irq; + struct v4l2_device v4l2_dev; + struct video_device *vdev_dec; + const struct v4l2_file_operations *v4l2_file_ops; + const struct v4l2_ioctl_ops *v4l2_ioctl_ops; +}; + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c new file mode 100644 index 0000000..2e93118 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -0,0 +1,111 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include + +#include "iris_core.h" + +static int iris_register_video_device(struct iris_core *core) +{ + struct video_device *vdev; + int ret; + + vdev = video_device_alloc(); + if (!vdev) + return -ENOMEM; + + strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name)); + vdev->release = video_device_release; + vdev->fops = core->v4l2_file_ops; + vdev->ioctl_ops = core->v4l2_ioctl_ops; + vdev->vfl_dir = VFL_DIR_M2M; + vdev->v4l2_dev = &core->v4l2_dev; + vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING; + + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); + if (ret) + goto err_vdev_release; + + core->vdev_dec = vdev; + video_set_drvdata(vdev, core); + + return ret; + +err_vdev_release: + video_device_release(vdev); + + return ret; +} + +static void iris_remove(struct platform_device *pdev) +{ + struct iris_core *core; + + core = platform_get_drvdata(pdev); + if (!core) + return; + + video_unregister_device(core->vdev_dec); + + v4l2_device_unregister(&core->v4l2_dev); +} + +static int iris_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct iris_core *core; + int ret; + + core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL); + if (!core) + return -ENOMEM; + core->dev = dev; + + core->reg_base = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(core->reg_base)) + return PTR_ERR(core->reg_base); + + core->irq = platform_get_irq(pdev, 0); + if (core->irq < 0) + return core->irq; + + ret = v4l2_device_register(dev, &core->v4l2_dev); + if (ret) + return ret; + + ret = iris_register_video_device(core); + if (ret) + goto err_v4l2_unreg; + + platform_set_drvdata(pdev, core); + + return ret; + +err_v4l2_unreg: + v4l2_device_unregister(&core->v4l2_dev); + + return ret; +} + +static const struct of_device_id iris_dt_match[] = { + { .compatible = "qcom,sm8550-iris", }, + { }, +}; +MODULE_DEVICE_TABLE(of, iris_dt_match); + +static struct platform_driver qcom_iris_driver = { + .probe = iris_probe, + .remove_new = iris_remove, + .driver = { + .name = "qcom-iris", + .of_match_table = iris_dt_match, + }, +}; + +module_platform_driver(qcom_iris_driver); +MODULE_DESCRIPTION("Qualcomm Iris video driver"); +MODULE_LICENSE("GPL"); From patchwork Mon Dec 18 11:32:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755801 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 267FE1A27E; Mon, 18 Dec 2023 11:33:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="NjoabHjr" Received: from pps.filterd (m0279872.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIBScQb021562; Mon, 18 Dec 2023 11:33:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=HWrsGR9dQBQYAECyvELmVJvsiibu4cOF7NAEek6IDMw=; b=Nj oabHjr+gMf44d+g9AkB7ma5F/Uo/Lbd5BOYAxAaPUBtkSV0JtCzDADxoR5adag40 xFeV5lhcuXy3n8XYa25wA/C63kkzlA0CXrSzXMN6zrNn1zUaMP05tm+cIkGfzNdo 2SCTIwRxP0h0SHQFzdjtoJEATOStcKj8poNU/Iyw9z+0BaA/QlX1MVMTNm0CYV2m AUNbgXQs9Hw2nFNv7FAqW72Rk1JuWedHM5A0JcdCO9W7oieCxfDGVRaEmqaGDGca rPGxGxmxIgKkQ5PXR6uO7tYMqz0+TBppZFtAbyHsjHwdNFwthYTZfZ+OUljKHmaZ cURi2vWBYEecfk3Fb5tA== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mjfr3a5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:33:10 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7t0030046; Mon, 18 Dec 2023 11:33:07 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyhe-1; Mon, 18 Dec 2023 11:33:07 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBVmqr028240; Mon, 18 Dec 2023 11:33:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6ki029964; Mon, 18 Dec 2023 11:33:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 3802122D8; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 10/34] media: iris: add PIL functionality for video firmware Date: Mon, 18 Dec 2023 17:02:05 +0530 Message-Id: <1702899149-21321-11-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: rlMvPhtwa_nFYWHeQKKLBMx4fwLr5wWF X-Proofpoint-ORIG-GUID: rlMvPhtwa_nFYWHeQKKLBMx4fwLr5wWF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 suspectscore=0 clxscore=1015 malwarescore=0 mlxscore=0 mlxlogscore=999 lowpriorityscore=0 bulkscore=0 impostorscore=0 priorityscore=1501 phishscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Load/unload firmware in memory via mdt loader. Firmware loading is part of core initialization and unloading is part of core de-initialization. This also changes the core states accordingly. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 10 ++- .../media/platform/qcom/vcodec/iris/iris_core.c | 70 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_core.h | 8 +++ .../media/platform/qcom/vcodec/iris/iris_helpers.c | 15 +++++ .../media/platform/qcom/vcodec/iris/iris_helpers.h | 13 ++++ drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 72 ++++++++++++++++++++++ drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 14 +++++ .../media/platform/qcom/vcodec/iris/iris_probe.c | 19 +++++- .../media/platform/qcom/vcodec/iris/iris_state.c | 9 ++- 9 files changed, 225 insertions(+), 5 deletions(-) create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_core.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_helpers.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index 59798e5d..74bd344 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -1,7 +1,11 @@ -iris-objs += ../hfi_queue.o +iris-objs += ../hfi_queue.o ../firmware.o iris-objs += iris_probe.o \ - resources.o \ - iris_state.o + iris_state.o \ + iris_core.o \ + iris_state.o \ + iris_helpers.o \ + iris_hfi.o \ + resources.o obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.c b/drivers/media/platform/qcom/vcodec/iris/iris_core.c new file mode 100644 index 0000000..ba8960d --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.c @@ -0,0 +1,70 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_core.h" +#include "iris_helpers.h" +#include "iris_hfi.h" +#include "iris_state.h" + +static int iris_core_deinit_locked(struct iris_core *core) +{ + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + + if (core->state == IRIS_CORE_DEINIT) + return 0; + + iris_hfi_core_deinit(core); + + iris_change_core_state(core, IRIS_CORE_DEINIT); + + return ret; +} + +int iris_core_deinit(struct iris_core *core) +{ + int ret; + + mutex_lock(&core->lock); + ret = iris_core_deinit_locked(core); + mutex_unlock(&core->lock); + + return ret; +} + +int iris_core_init(struct iris_core *core) +{ + int ret = 0; + + mutex_lock(&core->lock); + if (core_in_valid_state(core)) { + goto unlock; + } else if (core->state == IRIS_CORE_ERROR) { + ret = -EINVAL; + goto unlock; + } + + if (iris_change_core_state(core, IRIS_CORE_INIT_WAIT)) { + iris_change_core_state(core, IRIS_CORE_ERROR); + ret = -EINVAL; + goto unlock; + } + + ret = iris_hfi_core_init(core); + if (ret) { + iris_change_core_state(core, IRIS_CORE_ERROR); + dev_err(core->dev, "%s: core init failed\n", __func__); + iris_core_deinit_locked(core); + goto unlock; + } + +unlock: + mutex_unlock(&core->lock); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index 77124f9..2740ff1 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -11,6 +11,7 @@ #include "../hfi_queue.h" #include "iris_state.h" +#include "resources.h" /** * struct iris_core - holds core parameters valid for all instances @@ -36,6 +37,8 @@ * @message_queue: shared interface queue to receive responses from firmware * @debug_queue: shared interface queue to receive debug info from firmware * @sfr: SFR register memory + * @lock: a lock for this strucure + * @use_tz: a flag that suggests presence of trustzone */ struct iris_core { @@ -60,6 +63,11 @@ struct iris_core { struct iface_q_info message_queue; struct iface_q_info debug_queue; struct mem_desc sfr; + struct mutex lock; /* lock for core structure */ + unsigned int use_tz; }; +int iris_core_init(struct iris_core *core); +int iris_core_deinit(struct iris_core *core); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c new file mode 100644 index 0000000..22c706a --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_helpers.h" + +int check_core_lock(struct iris_core *core) +{ + bool fatal = !mutex_is_locked(&core->lock); + + WARN_ON(fatal); + + return fatal ? -EINVAL : 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h new file mode 100644 index 0000000..314a8d75 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_HELPERS_H_ +#define _IRIS_HELPERS_H_ + +#include "iris_core.h" + +int check_core_lock(struct iris_core *core); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c new file mode 100644 index 0000000..4f51a8c --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -0,0 +1,72 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "../firmware.h" +#include "iris_helpers.h" +#include "iris_hfi.h" + +#define CP_START 0 +#define CP_SIZE 0x25800000 +#define CP_NONPIXEL_START 0x01000000 +#define CP_NONPIXEL_SIZE 0x24800000 + +#define FW_NAME "vpu30_4v.mbn" +#define IRIS_PAS_ID 9 + +int iris_hfi_core_init(struct iris_core *core) +{ + phys_addr_t mem_phys = 0; + size_t mem_size = 0; + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + + ret = hfi_queue_init(core->dev, &core->iface_q_table, &core->sfr, + &core->command_queue, &core->message_queue, + &core->debug_queue, core); + if (ret) + goto error; + + core->use_tz = use_tz(core->dev); + if (!core->use_tz) + goto error; + + ret = load_fw(core->dev, FW_NAME, &mem_phys, &mem_size, + IRIS_PAS_ID, core->use_tz); + if (ret) + goto error; + + ret = auth_reset_fw(IRIS_PAS_ID); + if (ret) + goto error; + + ret = protect_secure_region(CP_START, CP_SIZE, CP_NONPIXEL_START, + CP_NONPIXEL_SIZE, IRIS_PAS_ID); + + return ret; + +error: + dev_err(core->dev, "%s(): failed with ret %d\n", __func__, ret); + + return ret; +} + +int iris_hfi_core_deinit(struct iris_core *core) +{ + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + + if (core->state == IRIS_CORE_DEINIT) + return 0; + + unload_fw(IRIS_PAS_ID); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h new file mode 100644 index 0000000..fcf9f28 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_HFI_H_ +#define _IRIS_HFI_H_ + +#include "iris_core.h" + +int iris_hfi_core_init(struct iris_core *core); +int iris_hfi_core_deinit(struct iris_core *core); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index fd349a3..f39b4aa 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -51,6 +51,7 @@ static void iris_remove(struct platform_device *pdev) if (!core) return; + iris_core_deinit(core); hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr, &core->command_queue, &core->message_queue, &core->debug_queue); @@ -58,6 +59,9 @@ static void iris_remove(struct platform_device *pdev) video_unregister_device(core->vdev_dec); v4l2_device_unregister(&core->v4l2_dev); + + mutex_destroy(&core->lock); + core->state = IRIS_CORE_DEINIT; } static int iris_probe(struct platform_device *pdev) @@ -72,6 +76,9 @@ static int iris_probe(struct platform_device *pdev) return -ENOMEM; core->dev = dev; + core->state = IRIS_CORE_DEINIT; + mutex_init(&core->lock); + core->reg_base = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(core->reg_base)) return PTR_ERR(core->reg_base); @@ -120,10 +127,20 @@ static int iris_probe(struct platform_device *pdev) goto err_vdev_unreg; } + ret = iris_core_init(core); + if (ret) { + dev_err_probe(core->dev, ret, "%s: core init failed\n", __func__); + goto err_queue_deinit; + } + return ret; +err_queue_deinit: + hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr, + &core->command_queue, &core->message_queue, + &core->debug_queue); err_vdev_unreg: - iris_unregister_video_device(core); + video_unregister_device(core->vdev_dec); err_v4l2_unreg: v4l2_device_unregister(&core->v4l2_dev); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c index 22557af..83bbc6b 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c @@ -4,6 +4,7 @@ */ #include "iris_core.h" +#include "iris_helpers.h" #include "iris_state.h" #define IRIS_STATE(name)[IRIS_CORE_##name] = "CORE_"#name @@ -52,6 +53,12 @@ static bool iris_allow_core_state_change(struct iris_core *core, int iris_change_core_state(struct iris_core *core, enum iris_core_state request_state) { + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + if (core->state == request_state) return 0; @@ -60,5 +67,5 @@ int iris_change_core_state(struct iris_core *core, core->state = request_state; - return 0; + return ret; } From patchwork Mon Dec 18 11:32:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755797 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E44371B28C; Mon, 18 Dec 2023 11:33:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="TpAnRsLP" Received: from pps.filterd (m0279870.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAs0cd008210; Mon, 18 Dec 2023 11:33:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=yjMHqh/n3Ch2ZrS6Ma1pzlwfHJQlBFURK7vxHKSwTCE=; b=Tp AnRsLPYnGNzagew7+f4F+ikwkLSvJbgC6YGH0LBwzl5IdRK3RgYy5c4GiPbuz+3B F0Won4jJaL0E5+QYnnYHrbG0x5nnHU3HAsswbHxtl87vhdM/cc0z+TuKv9B7hvrz iMrDOQVFVwYHKuknV2IiN9E+oI1X9fRz3cpYoUi9GJ7nCnuU0sKKHa8RTDivyub1 DbJ5iK3K5Vh4UjSNHPleDw7zmaNq2SC6YDdihgiD9aZC27WMIEKsItmvntxkcMZQ SdM7lfB6cFYZyjMlxcjlCQCPWfJK4vanJoTgUXKJZuyJHUunhOsTondB24OKh9Rt /jb/0KN+FCboEAxMedtA== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2m49g5h6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:33:12 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7u7030031; Mon, 18 Dec 2023 11:33:07 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyh6-1; Mon, 18 Dec 2023 11:33:07 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7dN029984; Mon, 18 Dec 2023 11:33:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6a6029970; Mon, 18 Dec 2023 11:33:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 3A92F22D9; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 11/34] media: iris: introduce packetization layer for creating HFI packets Date: Mon, 18 Dec 2023 17:02:06 +0530 Message-Id: <1702899149-21321-12-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: 4WoFG18BTgNUMyPf2ImxNsnxuVzUyCBA X-Proofpoint-GUID: 4WoFG18BTgNUMyPf2ImxNsnxuVzUyCBA X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 mlxlogscore=999 impostorscore=0 malwarescore=0 clxscore=1015 mlxscore=0 spamscore=0 suspectscore=0 bulkscore=0 priorityscore=1501 adultscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Host firmware interface (HFI) is well defined set of interfaces for communication between host driver and firmware. The command and responses are exchanged in form of packets. One or multiple packets are grouped under packet header. Each packet has packet type which describes the specific HFI and payload which holds the corresponding value for that HFI. Sys_init is the first packets sent to firmware, which initializes the firmware. Sys_image_version packet is to get the firmware version string. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 1 + .../media/platform/qcom/vcodec/iris/hfi_defines.h | 15 +++ .../media/platform/qcom/vcodec/iris/iris_core.h | 10 ++ drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 69 +++++++++++ .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 129 +++++++++++++++++++++ .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 71 ++++++++++++ .../media/platform/qcom/vcodec/iris/iris_probe.c | 5 + 7 files changed, 300 insertions(+) create mode 100644 drivers/media/platform/qcom/vcodec/iris/hfi_defines.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index 74bd344..a2d5d74 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -6,6 +6,7 @@ iris-objs += iris_probe.o \ iris_state.o \ iris_helpers.o \ iris_hfi.o \ + iris_hfi_packet.o \ resources.o obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h new file mode 100644 index 0000000..4c48c90 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _HFI_DEFINES_H_ +#define _HFI_DEFINES_H_ + +#define HFI_VIDEO_ARCH_LX 0x1 + +#define HFI_CMD_INIT 0x01000001 + +#define HFI_PROP_IMAGE_VERSION 0x03000001 + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index 2740ff1..de0cfef 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -39,6 +39,11 @@ * @sfr: SFR register memory * @lock: a lock for this strucure * @use_tz: a flag that suggests presence of trustzone + * @packet: pointer to packet from driver to fw + * @packet_size: size of packet + * @sys_init_id: id of sys init packet + * @header_id: id of packet header + * @packet_id: id of packet */ struct iris_core { @@ -65,6 +70,11 @@ struct iris_core { struct mem_desc sfr; struct mutex lock; /* lock for core structure */ unsigned int use_tz; + u8 *packet; + u32 packet_size; + u32 sys_init_id; + u32 header_id; + u32 packet_id; }; int iris_core_init(struct iris_core *core); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index 4f51a8c..fe16448 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -4,8 +4,67 @@ */ #include "../firmware.h" +#include "../hfi_queue.h" #include "iris_helpers.h" #include "iris_hfi.h" +#include "iris_hfi_packet.h" + +static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt) +{ + struct iface_q_info *q_info; + struct hfi_header *header; + u32 packet_size, rx_req; + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + + if (!core_in_valid_state(core)) + return -EINVAL; + + q_info = &core->command_queue; + if (!q_info || !q_info->q_array.kernel_vaddr || !pkt) { + dev_err(core->dev, "cannot write to shared CMD Q's\n"); + return -ENODATA; + } + + header = pkt; + packet_size = header->size; + + if (!write_queue(q_info, pkt, packet_size, &rx_req)) { + dev_err(core->dev, "queue full\n"); + return -ENODATA; + } + + return ret; +} + +static int sys_init(struct iris_core *core) +{ + int ret; + + ret = hfi_packet_sys_init(core, core->packet, core->packet_size); + if (ret) + return ret; + + ret = iris_hfi_queue_cmd_write(core, core->packet); + + return ret; +} + +static int sys_image_version(struct iris_core *core) +{ + int ret; + + ret = hfi_packet_image_version(core, core->packet, core->packet_size); + if (ret) + return ret; + + ret = iris_hfi_queue_cmd_write(core, core->packet); + + return ret; +} #define CP_START 0 #define CP_SIZE 0x25800000 @@ -46,6 +105,16 @@ int iris_hfi_core_init(struct iris_core *core) ret = protect_secure_region(CP_START, CP_SIZE, CP_NONPIXEL_START, CP_NONPIXEL_SIZE, IRIS_PAS_ID); + if (ret) + goto error; + + ret = sys_init(core); + if (ret) + goto error; + + ret = sys_image_version(core); + if (ret) + goto error; return ret; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c new file mode 100644 index 0000000..73bba07 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -0,0 +1,129 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_core.h" +#include "iris_hfi_packet.h" +#include "hfi_defines.h" + +static int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id, + u32 header_id) +{ + struct hfi_header *hdr = (struct hfi_header *)packet; + + if (!packet || packet_size < sizeof(*hdr)) + return -EINVAL; + + memset(hdr, 0, sizeof(*hdr)); + + hdr->size = sizeof(*hdr); + hdr->session_id = session_id; + hdr->header_id = header_id; + hdr->num_packets = 0; + + return 0; +} + +static int hfi_create_packet(u8 *packet, u32 packet_size, u32 pkt_type, + u32 pkt_flags, u32 payload_type, u32 port, + u32 packet_id, void *payload, u32 payload_size) +{ + struct hfi_header *hdr; + struct hfi_packet *pkt; + u32 pkt_size; + + if (!packet) + return -EINVAL; + + hdr = (struct hfi_header *)packet; + if (hdr->size < sizeof(*hdr)) + return -EINVAL; + + pkt = (struct hfi_packet *)(packet + hdr->size); + pkt_size = sizeof(*pkt) + payload_size; + if (packet_size < hdr->size + pkt_size) + return -EINVAL; + + memset(pkt, 0, pkt_size); + pkt->size = pkt_size; + pkt->type = pkt_type; + pkt->flags = pkt_flags; + pkt->payload_info = payload_type; + pkt->port = port; + pkt->packet_id = packet_id; + if (payload_size) + memcpy((u8 *)pkt + sizeof(*pkt), + payload, payload_size); + + hdr->num_packets++; + hdr->size += pkt->size; + + return 0; +} + +int hfi_packet_sys_init(struct iris_core *core, + u8 *pkt, u32 pkt_size) +{ + u32 payload = 0; + int ret; + + ret = hfi_create_header(pkt, pkt_size, + 0, + core->header_id++); + if (ret) + goto error; + + payload = HFI_VIDEO_ARCH_LX; + core->sys_init_id = core->packet_id++; + ret = hfi_create_packet(pkt, pkt_size, + HFI_CMD_INIT, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED | + HFI_HOST_FLAGS_NON_DISCARDABLE), + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->sys_init_id, + &payload, + sizeof(u32)); + if (ret) + goto error; + + return ret; + +error: + dev_err(core->dev, "%s: create sys init packet failed\n", __func__); + + return ret; +} + +int hfi_packet_image_version(struct iris_core *core, + u8 *pkt, u32 pkt_size) +{ + int ret; + + ret = hfi_create_header(pkt, pkt_size, + 0, + core->header_id++); + if (ret) + goto error; + + ret = hfi_create_packet(pkt, pkt_size, + HFI_PROP_IMAGE_VERSION, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED | + HFI_HOST_FLAGS_GET_PROPERTY), + HFI_PAYLOAD_NONE, + HFI_PORT_NONE, + core->packet_id++, + NULL, 0); + if (ret) + goto error; + + return ret; + +error: + dev_err(core->dev, "%s: create image version packet failed\n", __func__); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h new file mode 100644 index 0000000..e36612c --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -0,0 +1,71 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_HFI_PACKET_H_ +#define _IRIS_HFI_PACKET_H_ + +struct hfi_header { + u32 size; + u32 session_id; + u32 header_id; + u32 reserved[4]; + u32 num_packets; +}; + +struct hfi_packet { + u32 size; + u32 type; + u32 flags; + u32 payload_info; + u32 port; + u32 packet_id; + u32 reserved[2]; +}; + +enum hfi_packet_host_flags { + HFI_HOST_FLAGS_NONE = 0x00000000, + HFI_HOST_FLAGS_INTR_REQUIRED = 0x00000001, + HFI_HOST_FLAGS_RESPONSE_REQUIRED = 0x00000002, + HFI_HOST_FLAGS_NON_DISCARDABLE = 0x00000004, + HFI_HOST_FLAGS_GET_PROPERTY = 0x00000008, +}; + +enum hfi_packet_firmware_flags { + HFI_FW_FLAGS_NONE = 0x00000000, + HFI_FW_FLAGS_SUCCESS = 0x00000001, + HFI_FW_FLAGS_INFORMATION = 0x00000002, + HFI_FW_FLAGS_SESSION_ERROR = 0x00000004, + HFI_FW_FLAGS_SYSTEM_ERROR = 0x00000008, +}; + +enum hfi_packet_payload_info { + HFI_PAYLOAD_NONE = 0x00000000, + HFI_PAYLOAD_U32 = 0x00000001, + HFI_PAYLOAD_S32 = 0x00000002, + HFI_PAYLOAD_U64 = 0x00000003, + HFI_PAYLOAD_S64 = 0x00000004, + HFI_PAYLOAD_STRUCTURE = 0x00000005, + HFI_PAYLOAD_BLOB = 0x00000006, + HFI_PAYLOAD_STRING = 0x00000007, + HFI_PAYLOAD_Q16 = 0x00000008, + HFI_PAYLOAD_U32_ENUM = 0x00000009, + HFI_PAYLOAD_32_PACKED = 0x0000000a, + HFI_PAYLOAD_U32_ARRAY = 0x0000000b, + HFI_PAYLOAD_S32_ARRAY = 0x0000000c, + HFI_PAYLOAD_64_PACKED = 0x0000000d, +}; + +enum hfi_packet_port_type { + HFI_PORT_NONE = 0x00000000, + HFI_PORT_BITSTREAM = 0x00000001, + HFI_PORT_RAW = 0x00000002, +}; + +int hfi_packet_sys_init(struct iris_core *core, + u8 *pkt, u32 pkt_size); +int hfi_packet_image_version(struct iris_core *core, + u8 *pkt, u32 pkt_size); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index f39b4aa..570c64e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -79,6 +79,11 @@ static int iris_probe(struct platform_device *pdev) core->state = IRIS_CORE_DEINIT; mutex_init(&core->lock); + core->packet_size = IFACEQ_CORE_PKT_SIZE; + core->packet = devm_kzalloc(core->dev, core->packet_size, GFP_KERNEL); + if (!core->packet) + return -ENOMEM; + core->reg_base = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(core->reg_base)) return PTR_ERR(core->reg_base); From patchwork Mon Dec 18 11:32:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755799 Received: from mx0b-0031df01.pphosted.com (mx0b-0031df01.pphosted.com [205.220.180.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B398E1B286; Mon, 18 Dec 2023 11:33:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="LQspHlPT" Received: from pps.filterd (m0279872.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIArsDF026901; Mon, 18 Dec 2023 11:33:12 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=4sLJrCCYfSes228+tKDxIKdmEKAMGeK6QCzBbNNSTD4=; b=LQ spHlPT700/t/JailUFCIpFAuZ27YLa0mxbQkoPX1K43MbR0CO2wNOCp7wGk5zEUS xj7NVjRh0Fg4vsK0lHtNU1fONVAiCokTCuAZ0cSKfyq+g7B9wX7XZDKW78hzl2UQ a+11E3U5ZpKYswLbYh1B3QwoPfgXf4CaJJpxd8xwIIRRM0C/s+fO446XBvBAQRB4 Z87F/R0b1NDH+28ZPL+MM1SdYekXawK8K10nit+KVyDvgIoBUm5YCC302357tkXE a4wh+Lj4zsxmFEJLg20jVR7Zr36QdPk7DSpqh3xiElb0EYrI0jQtn1UO79tSm2EC OOyM/9Zzy1EkS2pwM2bA== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mjfr3a7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:33:12 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5MD029943; Mon, 18 Dec 2023 11:33:09 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyhn-1; Mon, 18 Dec 2023 11:33:07 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX72p029995; Mon, 18 Dec 2023 11:33:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6B1029969; Mon, 18 Dec 2023 11:33:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 3CF6D22F5; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 12/34] media: iris: add video processing unit(VPU) specific register handling Date: Mon, 18 Dec 2023 17:02:07 +0530 Message-Id: <1702899149-21321-13-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-GUID: rD8SNUoVX7ts_0OH9eqoGmsZ6moBsFK6 X-Proofpoint-ORIG-GUID: rD8SNUoVX7ts_0OH9eqoGmsZ6moBsFK6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 suspectscore=0 clxscore=1015 malwarescore=0 mlxscore=0 mlxlogscore=999 lowpriorityscore=0 bulkscore=0 impostorscore=0 priorityscore=1501 phishscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Registers are defined differently for different VPUs. Define ops for VPU specific handling to accommodate different VPUs. Implement boot sequence of firmware and interrupt programming. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 4 +- .../media/platform/qcom/vcodec/iris/iris_core.h | 3 + drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 7 + .../media/platform/qcom/vcodec/iris/iris_probe.c | 7 + .../media/platform/qcom/vcodec/iris/vpu_common.c | 71 +++++++++ .../media/platform/qcom/vcodec/iris/vpu_common.h | 32 ++++ .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 166 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/vpu_iris3.h | 13 ++ 8 files changed, 302 insertions(+), 1 deletion(-) create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_common.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index a2d5d74..90241b5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -7,6 +7,8 @@ iris-objs += iris_probe.o \ iris_helpers.o \ iris_hfi.o \ iris_hfi_packet.o \ - resources.o + resources.o \ + vpu_common.o \ + vpu_iris3.o obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index de0cfef..64678fd 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -12,6 +12,7 @@ #include "../hfi_queue.h" #include "iris_state.h" #include "resources.h" +#include "vpu_common.h" /** * struct iris_core - holds core parameters valid for all instances @@ -44,6 +45,7 @@ * @sys_init_id: id of sys init packet * @header_id: id of packet header * @packet_id: id of packet + * @vpu_ops: a pointer to vpu ops */ struct iris_core { @@ -75,6 +77,7 @@ struct iris_core { u32 sys_init_id; u32 header_id; u32 packet_id; + const struct vpu_ops *vpu_ops; }; int iris_core_init(struct iris_core *core); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index fe16448..7b3cbbc 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -8,6 +8,7 @@ #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_hfi_packet.h" +#include "vpu_common.h" static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt) { @@ -33,6 +34,8 @@ static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt) packet_size = header->size; if (!write_queue(q_info, pkt, packet_size, &rx_req)) { + call_vpu_op(core, raise_interrupt, core); + } else { dev_err(core->dev, "queue full\n"); return -ENODATA; } @@ -108,6 +111,10 @@ int iris_hfi_core_init(struct iris_core *core) if (ret) goto error; + ret = call_vpu_op(core, boot_firmware, core); + if (ret) + goto error; + ret = sys_init(core); if (ret) goto error; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index 570c64e..773481f 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -92,6 +92,13 @@ static int iris_probe(struct platform_device *pdev) if (core->irq < 0) return core->irq; + ret = init_vpu(core); + if (ret) { + dev_err_probe(core->dev, ret, + "%s: init vpu failed with %d\n", __func__, ret); + return ret; + } + ret = init_resources(core); if (ret) { dev_err_probe(core->dev, ret, diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.c b/drivers/media/platform/qcom/vcodec/iris/vpu_common.c new file mode 100644 index 0000000..3282510 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.c @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "vpu_iris3.h" +#include "iris_core.h" +#include "iris_helpers.h" +#include "vpu_common.h" + +int write_register(struct iris_core *core, u32 reg, u32 value) +{ + void __iomem *base_addr; + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + + base_addr = core->reg_base; + base_addr += reg; + writel_relaxed(value, base_addr); + + /* Make sure value is written into the register */ + wmb(); + + return ret; +} + +int read_register(struct iris_core *core, u32 reg, u32 *value) +{ + void __iomem *base_addr; + + base_addr = core->reg_base; + + *value = readl_relaxed(base_addr + reg); + + /* Make sure value is read correctly from the register */ + rmb(); + + return 0; +} + +static const struct compat_handle compat_handle[] = { + { + .compat = "qcom,sm8550-iris", + .init = init_iris3, + }, +}; + +int init_vpu(struct iris_core *core) +{ + struct device *dev = NULL; + int i, ret = 0; + + dev = core->dev; + + for (i = 0; i < ARRAY_SIZE(compat_handle); i++) { + if (of_device_is_compatible(dev->of_node, compat_handle[i].compat)) { + ret = compat_handle[i].init(core); + if (ret) + return ret; + break; + } + } + + if (i == ARRAY_SIZE(compat_handle)) + return -EINVAL; + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h new file mode 100644 index 0000000..790496a --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _VPU_COMMON_H_ +#define _VPU_COMMON_H_ + +#include + +struct iris_core; + +#define call_vpu_op(d, op, ...) \ + (((d) && (d)->vpu_ops && (d)->vpu_ops->op) ? \ + ((d)->vpu_ops->op(__VA_ARGS__)) : 0) + +struct compat_handle { + const char *compat; + int (*init)(struct iris_core *core); +}; + +struct vpu_ops { + int (*boot_firmware)(struct iris_core *core); + int (*raise_interrupt)(struct iris_core *core); +}; + +int init_vpu(struct iris_core *core); + +int write_register(struct iris_core *core, u32 reg, u32 value); +int read_register(struct iris_core *core, u32 reg, u32 *value); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c new file mode 100644 index 0000000..95bf223 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c @@ -0,0 +1,166 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include + +#include "vpu_iris3.h" + +#define VIDEO_ARCH_LX 1 + +#define CPU_BASE_OFFS_IRIS3 0x000A0000 + +#define CPU_CS_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3) +#define CPU_IC_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3) + +#define CPU_CS_VCICMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x24) +#define CPU_CS_VCICMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x28) + +/* HFI_CTRL_INIT */ +#define CPU_CS_SCIACMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x48) + +/* HFI_CTRL_STATUS */ +#define CPU_CS_SCIACMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x4C) +#define CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3 0x40000000 + +#define CPU_CS_H2XSOFTINTEN_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x148) + +#define CPU_CS_X2RPMH_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x168) + +/* UC_REGION_ADDR */ +#define CPU_CS_SCIBARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x64) + +/* UC_REGION_ADDR */ +#define CPU_CS_SCIBARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x68) + +/* HFI_QTBL_INFO */ +#define CPU_CS_SCIACMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x50) + +/* HFI_QTBL_ADDR */ +#define CPU_CS_SCIACMDARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x54) + +/* SFR_ADDR */ +#define CPU_CS_SCIBCMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x5C) + +#define UC_REGION_ADDR_IRIS3 CPU_CS_SCIBARG1_IRIS3 +#define UC_REGION_SIZE_IRIS3 CPU_CS_SCIBARG2_IRIS3 + +#define QTBL_INFO_IRIS3 CPU_CS_SCIACMDARG1_IRIS3 +#define QTBL_ADDR_IRIS3 CPU_CS_SCIACMDARG2_IRIS3 + +#define SFR_ADDR_IRIS3 CPU_CS_SCIBCMD_IRIS3 + +#define CTRL_INIT_IRIS3 CPU_CS_SCIACMD_IRIS3 + +#define CTRL_STATUS_IRIS3 CPU_CS_SCIACMDARG0_IRIS3 +#define CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 0xfe +#define CTRL_ERROR_STATUS__M_IRIS3 \ + CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 + +#define CPU_IC_SOFTINT_IRIS3 (CPU_IC_BASE_OFFS_IRIS3 + 0x150) +#define CPU_IC_SOFTINT_H2A_SHFT_IRIS3 0x0 + +static int setup_ucregion_memory_map_iris3(struct iris_core *core) +{ + int ret; + u32 value; + + value = (u32)core->iface_q_table.device_addr; + ret = write_register(core, UC_REGION_ADDR_IRIS3, value); + if (ret) + return ret; + + value = SHARED_QSIZE; + ret = write_register(core, UC_REGION_SIZE_IRIS3, value); + if (ret) + return ret; + + value = (u32)core->iface_q_table.device_addr; + ret = write_register(core, QTBL_ADDR_IRIS3, value); + if (ret) + return ret; + + ret = write_register(core, QTBL_INFO_IRIS3, 0x01); + if (ret) + return ret; + + value = (u32)((u64)core->iface_q_table.kernel_vaddr); + ret = write_register(core, CPU_CS_VCICMDARG0_IRIS3, value); + if (ret) + return ret; + + value = (u32)((u64)core->iface_q_table.kernel_vaddr >> 32); + ret = write_register(core, CPU_CS_VCICMDARG1_IRIS3, value); + if (ret) + return ret; + + if (core->sfr.device_addr) { + value = (u32)core->sfr.device_addr + VIDEO_ARCH_LX; + ret = write_register(core, SFR_ADDR_IRIS3, value); + if (ret) + return ret; + } + + return ret; +} + +static int boot_firmware_iris3(struct iris_core *core) +{ + u32 ctrl_init = 0, ctrl_status = 0, count = 0, max_tries = 1000; + int ret; + + ret = setup_ucregion_memory_map_iris3(core); + if (ret) + return ret; + + ctrl_init = BIT(0); + + ret = write_register(core, CTRL_INIT_IRIS3, ctrl_init); + if (ret) + return ret; + + while (!ctrl_status && count < max_tries) { + ret = read_register(core, CTRL_STATUS_IRIS3, &ctrl_status); + if (ret) + return ret; + + if ((ctrl_status & CTRL_ERROR_STATUS__M_IRIS3) == 0x4) { + dev_err(core->dev, "invalid setting for UC_REGION\n"); + break; + } + + usleep_range(50, 100); + count++; + } + + if (count >= max_tries) { + dev_err(core->dev, "Error booting up vidc firmware\n"); + return -ETIME; + } + + ret = write_register(core, CPU_CS_H2XSOFTINTEN_IRIS3, 0x1); + if (ret) + return ret; + + ret = write_register(core, CPU_CS_X2RPMH_IRIS3, 0x0); + + return ret; +} + +static int raise_interrupt_iris3(struct iris_core *core) +{ + return write_register(core, CPU_IC_SOFTINT_IRIS3, 1 << CPU_IC_SOFTINT_H2A_SHFT_IRIS3); +} + +static const struct vpu_ops iris3_ops = { + .boot_firmware = boot_firmware_iris3, + .raise_interrupt = raise_interrupt_iris3, +}; + +int init_iris3(struct iris_core *core) +{ + core->vpu_ops = &iris3_ops; + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h new file mode 100644 index 0000000..1424a5f --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2021-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _VPU_IRIS3_H_ +#define _VPU_IRIS3_H_ + +#include "iris_core.h" + +int init_iris3(struct iris_core *core); + +#endif From patchwork Mon Dec 18 11:32:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755789 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AA19B39AE0; Mon, 18 Dec 2023 11:40:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="VwhnCAj2" Received: from pps.filterd (m0279866.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAssR1018364; Mon, 18 Dec 2023 11:40:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=Fn5MQyElIA/ne+j5QqgL6eLBCk/yCOj7dBOuGhLSDLY=; b=Vw hnCAj2m/ZguBQ/kyO8yLolpbgB7v3QRrs+7APsLIez2aX7o6FUIFxdihm3phRqyR 5zfSWPSS67V9jM3mPEzDfXg1gYDoPBUy8b9YH65wngMYqlR4I1HsU/57dLvzDw8V w1U3qQf1KhairlIFbIu+oVdvK9Sveqwn+XN7kmuFHRjAWihskIQrPDIg9Y/rK53D Xq1N7lzzsDF+CdZ2+sh7OaPFm4hL2OIeTkVNTCjS4hGaiI+K7u37mLcNk+kE7XY8 uxxQJcU5DiUHceXFjVl8IHIZhdIExG72pDt1yHFQshM0mMyhevQ46LwP8ObFqcay D6HssxSZxBptbxHF8cyA== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v156dm7qc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:04 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX6PO029956; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ad-4; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX77k029983; Mon, 18 Dec 2023 11:33:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6Iw029963; Mon, 18 Dec 2023 11:33:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 420872305; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 14/34] media: iris: implement iris v4l2 file ops Date: Mon, 18 Dec 2023 17:02:09 +0530 Message-Id: <1702899149-21321-15-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: VPU_gMY1KiK2LJPhnV5D2jrAguq5PQvN X-Proofpoint-GUID: VPU_gMY1KiK2LJPhnV5D2jrAguq5PQvN X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 adultscore=0 lowpriorityscore=0 bulkscore=0 impostorscore=0 mlxscore=0 spamscore=0 priorityscore=1501 suspectscore=0 phishscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Vikash Garodia Implements iris v4l2 file ops - open, close and poll. Open: Configure the vb2 queue and v4l2 file handler. Initialize a video instance and add the instance to core instance list. Open a session in video firmware via HFI_CMD_OPEN. Close: De-initialize the instance and remove it from core instance list. Close the session in firmware via HFI_CMD_CLOSE. Signed-off-by: Vikash Garodia Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 2 + .../media/platform/qcom/vcodec/iris/hfi_defines.h | 2 + .../media/platform/qcom/vcodec/iris/iris_common.h | 26 ++ .../media/platform/qcom/vcodec/iris/iris_core.c | 42 ++- .../media/platform/qcom/vcodec/iris/iris_core.h | 8 + .../media/platform/qcom/vcodec/iris/iris_helpers.c | 62 +++++ .../media/platform/qcom/vcodec/iris/iris_helpers.h | 2 + drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 115 +++++++- drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 3 + .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 28 ++ .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 7 + .../platform/qcom/vcodec/iris/iris_instance.h | 48 ++++ .../media/platform/qcom/vcodec/iris/iris_probe.c | 10 + .../media/platform/qcom/vcodec/iris/iris_vdec.c | 47 ++++ .../media/platform/qcom/vcodec/iris/iris_vdec.h | 14 + .../media/platform/qcom/vcodec/iris/iris_vidc.c | 298 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_vidc.h | 15 ++ 17 files changed, 719 insertions(+), 10 deletions(-) create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_common.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_instance.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vdec.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vidc.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index c50e3241..3c076d0 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -3,6 +3,8 @@ iris-objs += ../hfi_queue.o ../firmware.o iris-objs += iris_probe.o \ iris_state.o \ iris_core.o \ + iris_vidc.o \ + iris_vdec.o \ iris_state.o \ iris_helpers.o \ iris_hfi.o \ diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index fb383b2..423ba1a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -9,6 +9,8 @@ #define HFI_VIDEO_ARCH_LX 0x1 #define HFI_CMD_INIT 0x01000001 +#define HFI_CMD_OPEN 0x01000003 +#define HFI_CMD_CLOSE 0x01000004 #define HFI_PROP_IMAGE_VERSION 0x03000001 diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h new file mode 100644 index 0000000..3e4dd71 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#ifndef _IRIS_COMMON_H_ +#define _IRIS_COMMON_H_ + +#include +#include +#include +#include + +#define INPUT_MPLANE V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE +#define OUTPUT_MPLANE V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE +#define DEFAULT_WIDTH 320 +#define DEFAULT_HEIGHT 240 +#define DEFAULT_BUF_SIZE 115200 + +enum signal_session_response { + SIGNAL_CMD_STOP_INPUT = 0, + SIGNAL_CMD_STOP_OUTPUT, + SIGNAL_CMD_CLOSE, + MAX_SIGNAL, +}; + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.c b/drivers/media/platform/qcom/vcodec/iris/iris_core.c index ba8960d..174fad9 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.c @@ -3,12 +3,14 @@ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include + #include "iris_core.h" #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_state.h" -static int iris_core_deinit_locked(struct iris_core *core) +int iris_core_deinit_locked(struct iris_core *core) { int ret; @@ -68,3 +70,41 @@ int iris_core_init(struct iris_core *core) return ret; } + +int iris_core_init_wait(struct iris_core *core) +{ + const int interval = 10; + int max_tries, count = 0, ret = 0; + + mutex_lock(&core->lock); + if (!core_in_valid_state(core)) { + ret = -EINVAL; + goto unlock; + } + + if (core->state == IRIS_CORE_INIT) + goto unlock; + + max_tries = core->cap[HW_RESPONSE_TIMEOUT].value / interval; + while (count < max_tries) { + if (core->state != IRIS_CORE_INIT_WAIT) + break; + msleep(interval); + count++; + } + + if (core->state == IRIS_CORE_INIT) { + ret = 0; + goto unlock; + } else { + iris_change_core_state(core, IRIS_CORE_ERROR); + iris_core_deinit_locked(core); + ret = -EINVAL; + goto unlock; + } + +unlock: + mutex_unlock(&core->lock); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index 3c8497a..b47520a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -33,6 +33,8 @@ * @clk_count: count of iris clocks * @reset_tbl: table of iris reset clocks * @reset_count: count of iris reset clocks + * @vb2_ops: iris vb2 ops + * @vb2_mem_ops: iris vb2 memory ops * @state: current state of core * @iface_q_table: Interface queue table memory * @command_queue: shared interface queue to send commands to firmware @@ -49,6 +51,7 @@ * @vpu_ops: a pointer to vpu ops * @platform_data: a structure for platform data * @cap: an array for supported core capabilities + * @instances: a list_head of all instances */ struct iris_core { @@ -67,6 +70,8 @@ struct iris_core { u32 clk_count; struct reset_info *reset_tbl; u32 reset_count; + const struct vb2_ops *vb2_ops; + struct vb2_mem_ops *vb2_mem_ops; enum iris_core_state state; struct mem_desc iface_q_table; struct iface_q_info command_queue; @@ -83,9 +88,12 @@ struct iris_core { const struct vpu_ops *vpu_ops; struct platform_data *platform_data; struct plat_core_cap cap[CORE_CAP_MAX + 1]; + struct list_head instances; }; int iris_core_init(struct iris_core *core); +int iris_core_init_wait(struct iris_core *core); int iris_core_deinit(struct iris_core *core); +int iris_core_deinit_locked(struct iris_core *core); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index c31dfd5..872268d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -4,6 +4,8 @@ */ #include "iris_helpers.h" +#include "iris_hfi.h" +#include "iris_instance.h" int check_core_lock(struct iris_core *core) { @@ -32,3 +34,63 @@ int iris_init_core_caps(struct iris_core *core) return 0; } + +static int process_inst_timeout(struct iris_inst *inst) +{ + struct iris_inst *instance; + struct iris_core *core; + bool found = false; + int ret = 0; + + core = inst->core; + + mutex_lock(&core->lock); + list_for_each_entry(instance, &core->instances, list) { + if (instance == inst) { + found = true; + break; + } + } + if (!found) { + ret = -EINVAL; + goto unlock; + } + + iris_change_core_state(core, IRIS_CORE_ERROR); + + iris_core_deinit_locked(core); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + +int close_session(struct iris_inst *inst) +{ + u32 hw_response_timeout_val; + bool wait_for_response; + struct iris_core *core; + int ret; + + core = inst->core; + hw_response_timeout_val = core->cap[HW_RESPONSE_TIMEOUT].value; + wait_for_response = true; + ret = iris_hfi_session_close(inst); + if (ret) + wait_for_response = false; + + kfree(inst->packet); + inst->packet = NULL; + + if (wait_for_response) { + ret = wait_for_completion_timeout(&inst->completions[SIGNAL_CMD_CLOSE], + msecs_to_jiffies(hw_response_timeout_val)); + if (!ret) { + ret = -ETIMEDOUT; + process_inst_timeout(inst); + } + } + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index 4c7ddbf..b9a6485 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -7,8 +7,10 @@ #define _IRIS_HELPERS_H_ #include "iris_core.h" +#include "iris_instance.h" int check_core_lock(struct iris_core *core); int iris_init_core_caps(struct iris_core *core); +int close_session(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index b0390b542..d144ae5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -8,6 +8,7 @@ #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_hfi_packet.h" +#include "hfi_defines.h" #include "vpu_common.h" static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt) @@ -43,6 +44,27 @@ static int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt) return ret; } +static bool validate_session(struct iris_core *core, + struct iris_inst *inst) +{ + struct iris_inst *temp; + bool valid = false; + int ret; + + ret = check_core_lock(core); + if (ret) + return false; + + list_for_each_entry(temp, &core->instances, list) { + if (temp == inst) { + valid = true; + break; + } + } + + return valid; +} + static int sys_init(struct iris_core *core) { int ret; @@ -69,14 +91,6 @@ static int sys_image_version(struct iris_core *core) return ret; } -#define CP_START 0 -#define CP_SIZE 0x25800000 -#define CP_NONPIXEL_START 0x01000000 -#define CP_NONPIXEL_SIZE 0x24800000 - -#define FW_NAME "vpu30_4v.mbn" -#define IRIS_PAS_ID 9 - int iris_hfi_core_init(struct iris_core *core) { u32 cp_nonpixel_start, cp_nonpixel_size; @@ -152,7 +166,90 @@ int iris_hfi_core_deinit(struct iris_core *core) if (core->state == IRIS_CORE_DEINIT) return 0; - unload_fw(IRIS_PAS_ID); + unload_fw(core->platform_data->pas_id); + + return ret; +} + +int iris_hfi_session_open(struct iris_inst *inst) +{ + struct iris_core *core; + int ret; + + inst->packet_size = 4096; + inst->packet = kzalloc(inst->packet_size, GFP_KERNEL); + if (!inst->packet) + return -ENOMEM; + + core = inst->core; + mutex_lock(&core->lock); + + if (!validate_session(core, inst)) { + ret = -EINVAL; + goto fail_free_packet; + } + + ret = hfi_packet_session_command(inst, + HFI_CMD_OPEN, + HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED, + HFI_PORT_NONE, + 0, + HFI_PAYLOAD_U32, + &inst->session_id, + sizeof(u32)); + if (ret) + goto fail_free_packet; + + ret = iris_hfi_queue_cmd_write(core, inst->packet); + if (ret) + goto fail_free_packet; + + mutex_unlock(&core->lock); + + return ret; + +fail_free_packet: + kfree(inst->packet); + inst->packet = NULL; + mutex_unlock(&core->lock); + + return ret; +} + +int iris_hfi_session_close(struct iris_inst *inst) +{ + struct iris_core *core; + int ret; + + if (!inst->packet) + return -EINVAL; + + core = inst->core; + mutex_lock(&core->lock); + + if (!validate_session(core, inst)) { + ret = -EINVAL; + goto unlock; + } + + ret = hfi_packet_session_command(inst, + HFI_CMD_CLOSE, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED | + HFI_HOST_FLAGS_NON_DISCARDABLE), + HFI_PORT_NONE, + inst->session_id, + HFI_PAYLOAD_NONE, + NULL, + 0); + if (ret) + goto unlock; + + ret = iris_hfi_queue_cmd_write(inst->core, inst->packet); + +unlock: + mutex_unlock(&core->lock); return ret; } diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h index fcf9f28..8a057cc 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h @@ -6,9 +6,12 @@ #ifndef _IRIS_HFI_H_ #define _IRIS_HFI_H_ +#include "iris_instance.h" #include "iris_core.h" int iris_hfi_core_init(struct iris_core *core); int iris_hfi_core_deinit(struct iris_core *core); +int iris_hfi_session_open(struct iris_inst *inst); +int iris_hfi_session_close(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c index f31dd84..1ed572e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -211,3 +211,31 @@ int hfi_packet_image_version(struct iris_core *core, return ret; } + +int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type, + u32 flags, u32 port, u32 session_id, + u32 payload_type, void *payload, + u32 payload_size) +{ + struct iris_core *core; + int ret; + + core = inst->core; + + ret = hfi_create_header(inst->packet, inst->packet_size, + session_id, core->header_id++); + if (ret) + return ret; + + ret = hfi_create_packet(inst->packet, + inst->packet_size, + pkt_type, + flags, + payload_type, + port, + core->packet_id++, + payload, + payload_size); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h index e36612c..9e476e9 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -6,6 +6,9 @@ #ifndef _IRIS_HFI_PACKET_H_ #define _IRIS_HFI_PACKET_H_ +#include "iris_core.h" +#include "iris_instance.h" + struct hfi_header { u32 size; u32 session_id; @@ -67,5 +70,9 @@ int hfi_packet_sys_init(struct iris_core *core, u8 *pkt, u32 pkt_size); int hfi_packet_image_version(struct iris_core *core, u8 *pkt, u32 pkt_size); +int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type, + u32 flags, u32 port, u32 session_id, + u32 payload_type, void *payload, + u32 payload_size); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h new file mode 100644 index 0000000..1bbb533 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_INSTANCE_H_ +#define _IRIS_INSTANCE_H_ + +#include + +#include "iris_core.h" +#include "iris_common.h" + +/** + * struct iris_inst - holds per video instance parameters + * + * @list: used for attach an instance to the core + * @core: pointer to core structure + * @session_id: id of current video session + * @vb2q_src: source vb2 queue + * @vb2q_dst: destination vb2 queue + * @ctx_q_lock: lock to serialize queues related ioctls + * @fh: reference of v4l2 file handler + * @fmt_src: structure of v4l2_format for source + * @fmt_dst: structure of v4l2_format for destination + * @ctrl_handler: reference of v4l2 ctrl handler + * @packet: HFI packet + * @packet_size: HFI packet size + * @completions: structure of signal completions + */ + +struct iris_inst { + struct list_head list; + struct iris_core *core; + u32 session_id; + struct vb2_queue *vb2q_src; + struct vb2_queue *vb2q_dst; + struct mutex ctx_q_lock;/* lock to serialize queues related ioctls */ + struct v4l2_fh fh; + struct v4l2_format *fmt_src; + struct v4l2_format *fmt_dst; + struct v4l2_ctrl_handler ctrl_handler; + void *packet; + u32 packet_size; + struct completion completions[MAX_SIGNAL]; +}; + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index 9d6a6c5..4f20da8 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -11,6 +11,7 @@ #include "iris_core.h" #include "iris_helpers.h" #include "resources.h" +#include "iris_vidc.h" static int iris_register_video_device(struct iris_core *core) { @@ -85,6 +86,8 @@ static int iris_probe(struct platform_device *pdev) if (!core->packet) return -ENOMEM; + INIT_LIST_HEAD(&core->instances); + core->reg_base = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(core->reg_base)) return PTR_ERR(core->reg_base); @@ -107,6 +110,13 @@ static int iris_probe(struct platform_device *pdev) return ret; } + ret = init_ops(core); + if (ret) { + dev_err_probe(core->dev, ret, + "%s: init ops failed with %d\n", __func__, ret); + return ret; + } + ret = init_resources(core); if (ret) { dev_err_probe(core->dev, ret, diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c new file mode 100644 index 0000000..984be34 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -0,0 +1,47 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_common.h" +#include "iris_vdec.h" + +void vdec_inst_init(struct iris_inst *inst) +{ + struct v4l2_format *f; + + inst->fmt_src = kzalloc(sizeof(*inst->fmt_src), GFP_KERNEL); + inst->fmt_dst = kzalloc(sizeof(*inst->fmt_dst), GFP_KERNEL); + inst->vb2q_src = kzalloc(sizeof(*inst->vb2q_src), GFP_KERNEL); + inst->vb2q_dst = kzalloc(sizeof(*inst->vb2q_dst), GFP_KERNEL); + + f = inst->fmt_src; + f->type = INPUT_MPLANE; + f->fmt.pix_mp.width = DEFAULT_WIDTH; + f->fmt.pix_mp.height = DEFAULT_HEIGHT; + f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_H264; + f->fmt.pix_mp.num_planes = 1; + f->fmt.pix_mp.plane_fmt[0].bytesperline = 0; + f->fmt.pix_mp.plane_fmt[0].sizeimage = DEFAULT_BUF_SIZE; + f->fmt.pix_mp.field = V4L2_FIELD_NONE; + + f = inst->fmt_dst; + f->type = OUTPUT_MPLANE; + f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_QC08C; + f->fmt.pix_mp.width = ALIGN(DEFAULT_WIDTH, 128); + f->fmt.pix_mp.height = ALIGN(DEFAULT_HEIGHT, 32); + f->fmt.pix_mp.num_planes = 1; + f->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(DEFAULT_WIDTH, 128); + f->fmt.pix_mp.plane_fmt[0].sizeimage = DEFAULT_BUF_SIZE; + f->fmt.pix_mp.field = V4L2_FIELD_NONE; + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; +} + +void vdec_inst_deinit(struct iris_inst *inst) +{ + kfree(inst->fmt_dst); + kfree(inst->fmt_src); +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h new file mode 100644 index 0000000..dc8f43f --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_VDEC_H_ +#define _IRIS_VDEC_H_ + +#include "iris_instance.h" + +void vdec_inst_init(struct iris_inst *inst); +void vdec_inst_deinit(struct iris_inst *inst); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c new file mode 100644 index 0000000..d02da8b --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -0,0 +1,298 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_common.h" +#include "iris_helpers.h" +#include "iris_hfi.h" +#include "iris_instance.h" +#include "iris_vdec.h" +#include "iris_vidc.h" + +static int vidc_v4l2_fh_init(struct iris_inst *inst) +{ + struct iris_core *core; + + core = inst->core; + + if (inst->fh.vdev) + return -EINVAL; + + v4l2_fh_init(&inst->fh, core->vdev_dec); + inst->fh.ctrl_handler = &inst->ctrl_handler; + v4l2_fh_add(&inst->fh); + + return 0; +} + +static int vidc_v4l2_fh_deinit(struct iris_inst *inst) +{ + if (!inst->fh.vdev) + return 0; + + v4l2_fh_del(&inst->fh); + inst->fh.ctrl_handler = NULL; + v4l2_fh_exit(&inst->fh); + + return 0; +} + +static int vb2q_init(struct iris_inst *inst, + struct vb2_queue *q, enum v4l2_buf_type type) +{ + struct iris_core *core; + + core = inst->core; + + q->lock = &inst->ctx_q_lock; + q->type = type; + q->io_modes = VB2_MMAP | VB2_DMABUF; + q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + q->ops = core->vb2_ops; + q->mem_ops = core->vb2_mem_ops; + q->drv_priv = inst; + q->copy_timestamp = 1; + q->min_buffers_needed = 0; + return vb2_queue_init(q); +} + +static int vidc_vb2_queue_init(struct iris_inst *inst) +{ + int ret; + + ret = vb2q_init(inst, inst->vb2q_src, INPUT_MPLANE); + if (ret) + return ret; + + ret = vb2q_init(inst, inst->vb2q_dst, OUTPUT_MPLANE); + if (ret) + goto fail_vb2q_src_deinit; + + return ret; + +fail_vb2q_src_deinit: + vb2_queue_release(inst->vb2q_src); + + return ret; +} + +static int vidc_vb2_queue_deinit(struct iris_inst *inst) +{ + vb2_queue_release(inst->vb2q_src); + kfree(inst->vb2q_src); + inst->vb2q_src = NULL; + + vb2_queue_release(inst->vb2q_dst); + kfree(inst->vb2q_dst); + inst->vb2q_dst = NULL; + + return 0; +} + +static int vidc_add_session(struct iris_inst *inst) +{ + struct iris_core *core; + struct iris_inst *i; + u32 count = 0; + int ret = 0; + + core = inst->core; + + mutex_lock(&core->lock); + if (core->state != IRIS_CORE_INIT) { + ret = -EINVAL; + goto unlock; + } + list_for_each_entry(i, &core->instances, list) + count++; + + if (count < core->cap[MAX_SESSION_COUNT].value) + list_add_tail(&inst->list, &core->instances); + else + ret = -EAGAIN; +unlock: + mutex_unlock(&core->lock); + + return ret; +} + +static int vidc_remove_session(struct iris_inst *inst) +{ + struct iris_inst *i, *temp; + struct iris_core *core; + + core = inst->core; + + mutex_lock(&core->lock); + list_for_each_entry_safe(i, temp, &core->instances, list) { + if (i->session_id == inst->session_id) { + list_del_init(&i->list); + break; + } + } + mutex_unlock(&core->lock); + + return 0; +} + +static struct iris_inst *get_vidc_inst(struct file *filp, void *fh) +{ + if (!filp || !filp->private_data) + return NULL; + + return container_of(filp->private_data, + struct iris_inst, fh); +} + +int vidc_open(struct file *filp) +{ + struct iris_core *core = video_drvdata(filp); + struct iris_inst *inst = NULL; + int i = 0; + int ret; + + ret = iris_core_init(core); + if (ret) + return ret; + + ret = iris_core_init_wait(core); + if (ret) + return ret; + + inst = kzalloc(sizeof(*inst), GFP_KERNEL); + if (!inst) + return -ENOMEM; + + inst->core = core; + inst->session_id = hash32_ptr(inst); + mutex_init(&inst->ctx_q_lock); + for (i = 0; i < MAX_SIGNAL; i++) + init_completion(&inst->completions[i]); + + ret = vidc_add_session(inst); + if (ret) + goto fail_free_inst; + + ret = vidc_v4l2_fh_init(inst); + if (ret) + goto fail_remove_session; + + vdec_inst_init(inst); + + ret = vidc_vb2_queue_init(inst); + if (ret) + goto fail_inst_deinit; + + ret = iris_hfi_session_open(inst); + if (ret) { + dev_err(core->dev, "%s: session open failed\n", __func__); + goto fail_core_deinit; + } + filp->private_data = &inst->fh; + + return 0; + +fail_core_deinit: + iris_core_deinit(core); + vidc_vb2_queue_deinit(inst); +fail_inst_deinit: + vdec_inst_deinit(inst); + vidc_v4l2_fh_deinit(inst); +fail_remove_session: + vidc_remove_session(inst); +fail_free_inst: + mutex_destroy(&inst->ctx_q_lock); + kfree(inst); + + return ret; +} + +int vidc_close(struct file *filp) +{ + struct iris_inst *inst; + + inst = get_vidc_inst(filp, NULL); + if (!inst) + return -EINVAL; + + vdec_inst_deinit(inst); + close_session(inst); + vidc_vb2_queue_deinit(inst); + vidc_v4l2_fh_deinit(inst); + vidc_remove_session(inst); + mutex_destroy(&inst->ctx_q_lock); + kfree(inst); + + filp->private_data = NULL; + + return 0; +} + +static __poll_t get_poll_flags(struct iris_inst *inst, u32 plane) +{ + struct vb2_buffer *vb = NULL; + struct vb2_queue *q = NULL; + unsigned long flags = 0; + __poll_t poll = 0; + + if (plane == INPUT_MPLANE) + q = inst->vb2q_src; + else if (plane == OUTPUT_MPLANE) + q = inst->vb2q_dst; + + if (!q) + return EPOLLERR; + + spin_lock_irqsave(&q->done_lock, flags); + if (!list_empty(&q->done_list)) + vb = list_first_entry(&q->done_list, struct vb2_buffer, + done_entry); + if (vb && (vb->state == VB2_BUF_STATE_DONE || + vb->state == VB2_BUF_STATE_ERROR)) { + if (plane == OUTPUT_MPLANE) + poll |= EPOLLIN | EPOLLRDNORM; + else if (plane == INPUT_MPLANE) + poll |= EPOLLOUT | EPOLLWRNORM; + } + spin_unlock_irqrestore(&q->done_lock, flags); + + return poll; +} + +static __poll_t vidc_poll(struct file *filp, struct poll_table_struct *pt) +{ + struct iris_inst *inst; + __poll_t poll = 0; + + inst = get_vidc_inst(filp, NULL); + if (!inst) + return EPOLLERR; + + poll_wait(filp, &inst->fh.wait, pt); + poll_wait(filp, &inst->vb2q_src->done_wq, pt); + poll_wait(filp, &inst->vb2q_dst->done_wq, pt); + + if (v4l2_event_pending(&inst->fh)) + poll |= EPOLLPRI; + + poll |= get_poll_flags(inst, INPUT_MPLANE); + poll |= get_poll_flags(inst, OUTPUT_MPLANE); + + return poll; +} + +static const struct v4l2_file_operations v4l2_file_ops = { + .owner = THIS_MODULE, + .open = vidc_open, + .release = vidc_close, + .unlocked_ioctl = video_ioctl2, + .poll = vidc_poll, +}; + +int init_ops(struct iris_core *core) +{ + core->v4l2_file_ops = &v4l2_file_ops; + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.h b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.h new file mode 100644 index 0000000..aebff44 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_VIDC_H_ +#define _IRIS_VIDC_H_ + +#include "iris_core.h" + +int init_ops(struct iris_core *core); +int vidc_open(struct file *filp); +int vidc_close(struct file *filp); + +#endif From patchwork Mon Dec 18 11:32:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755794 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 10CE91B26E; Mon, 18 Dec 2023 11:36:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="QZrGMNdu" Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIBJih7018958; Mon, 18 Dec 2023 11:36:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=Ii26TUpLDcoDUZzoj7f1vyuG0tLLz7zFj9QofAAkMJc=; b=QZ rGMNdu31ybxjmQvSAmfYuBIR32xUGJN3EkhF4PeV+9cBGrOSAlplCfxg5g1WMim4 3tubvgSWRvaDsq19MlX+ucIlyF5r8oZe2D1KiKlqaNupN9CvdpX8hU53kqc3yR+m YILDLSvYRQ+EQqENs4CMbMLYyl13XQz+dg4EISp10+fkCSxZTzxpu8Aqd6AgHupf W8rISJ7d2es94YGHJYRQAyeBUP4X1caD5JeWyOhg+Ov9xW0Rz42VJ1TYRRV/kL0o dP1GZ/MYGWBMFseqzaPYMHya1RsOCD4dU+1EQgRtzr3ARFNvGuijYmb5+T95H538 l4KP/c+tyoctDXyoWjFw== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mb9g4my-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:36:12 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBVmVB028246; Mon, 18 Dec 2023 11:36:09 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyjr-1; Mon, 18 Dec 2023 11:33:09 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5Aq029911; Mon, 18 Dec 2023 11:33:09 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6q7029965; Mon, 18 Dec 2023 11:33:09 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 49C562308; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 17/34] media: iris: implement vb2_ops queue setup Date: Mon, 18 Dec 2023 17:02:12 +0530 Message-Id: <1702899149-21321-18-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: cSe5nSbz30KhCIGHqG7yaTukO9KC5Hzn X-Proofpoint-GUID: cSe5nSbz30KhCIGHqG7yaTukO9KC5Hzn X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 impostorscore=0 adultscore=0 mlxscore=0 clxscore=1015 spamscore=0 mlxlogscore=999 phishscore=0 lowpriorityscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Implement queue_setup vb2_ops. Calculate the buffer count and buffer size as par video hardware requirement and updates to client. Also, allocate the video driver buffers for output and capture plane. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 4 +- .../media/platform/qcom/vcodec/iris/iris_buffer.c | 179 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_buffer.h | 49 ++++++ .../media/platform/qcom/vcodec/iris/iris_common.h | 71 +++++++- .../media/platform/qcom/vcodec/iris/iris_helpers.c | 24 +++ .../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 +- .../platform/qcom/vcodec/iris/iris_instance.h | 6 + drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 71 ++++++++ drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 15 ++ .../media/platform/qcom/vcodec/iris/iris_vdec.c | 5 +- .../media/platform/qcom/vcodec/iris/iris_vidc.c | 19 ++- .../platform/qcom/vcodec/iris/platform_common.h | 16 +- 12 files changed, 442 insertions(+), 21 deletions(-) create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_buffer.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_vb2.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index 7fdee5b..a94e36b 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -1,9 +1,10 @@ -iris-objs += ../hfi_queue.o ../firmware.o +iris-objs += ../hfi_queue.o ../firmware.o ../buffers.o iris-objs += iris_probe.o \ iris_state.o \ iris_core.o \ iris_vidc.o \ + iris_vb2.o \ iris_vdec.o \ iris_state.o \ iris_ctrls.o \ @@ -11,6 +12,7 @@ iris-objs += iris_probe.o \ iris_hfi.o \ iris_hfi_response.o \ iris_hfi_packet.o \ + iris_buffer.o \ resources.o \ vpu_common.o \ vpu_iris3.o \ diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c new file mode 100644 index 0000000..b9cffbf --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c @@ -0,0 +1,179 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "../buffers.h" +#include "iris_buffer.h" +#include "iris_helpers.h" +#include "iris_instance.h" + +static int input_min_count(struct iris_inst *inst) +{ + return MIN_BUFFERS; +} + +static int output_min_count(struct iris_inst *inst) +{ + int output_min_count; + + switch (inst->codec) { + case H264: + case HEVC: + output_min_count = 4; + break; + case VP9: + output_min_count = 9; + break; + default: + output_min_count = 4; + break; + } + + return output_min_count; +} + +int iris_get_buf_min_count(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + switch (buffer_type) { + case BUF_INPUT: + return input_min_count(inst); + case BUF_OUTPUT: + return output_min_count(inst); + default: + return 0; + } +} + +static u32 input_buffer_size(struct iris_inst *inst) +{ + u32 base_res_mbs = NUM_MBS_4k; + u32 frame_size, num_mbs; + struct v4l2_format *f; + u32 div_factor = 1; + u32 codec; + + f = inst->fmt_src; + codec = f->fmt.pix_mp.pixelformat; + + num_mbs = get_mbpf(inst); + if (num_mbs > NUM_MBS_4k) { + div_factor = 4; + base_res_mbs = inst->cap[MBPF].value; + } else { + base_res_mbs = NUM_MBS_4k; + if (codec == V4L2_PIX_FMT_VP9) + div_factor = 1; + else + div_factor = 2; + } + + frame_size = base_res_mbs * MB_IN_PIXEL * 3 / 2 / div_factor; + + /* multiply by 10/8 (1.25) to get size for 10 bit case */ + if (codec == V4L2_PIX_FMT_VP9 || codec == V4L2_PIX_FMT_HEVC) + frame_size = frame_size + (frame_size >> 2); + + return ALIGN(frame_size, SZ_4K); +} + +static u32 output_buffer_size(struct iris_inst *inst) +{ + struct v4l2_format *f; + u32 size; + + f = inst->fmt_dst; + + size = video_raw_buffer_size(f->fmt.pix_mp.pixelformat, f->fmt.pix_mp.width, + f->fmt.pix_mp.height); + return size; +} + +int iris_get_buffer_size(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + switch (buffer_type) { + case BUF_INPUT: + return input_buffer_size(inst); + case BUF_OUTPUT: + return output_buffer_size(inst); + default: + return 0; + } +} + +struct iris_buffers *iris_get_buffer_list(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + switch (buffer_type) { + case BUF_INPUT: + return &inst->buffers.input; + case BUF_OUTPUT: + return &inst->buffers.output; + case BUF_READ_ONLY: + return &inst->buffers.read_only; + case BUF_BIN: + return &inst->buffers.bin; + case BUF_ARP: + return &inst->buffers.arp; + case BUF_COMV: + return &inst->buffers.comv; + case BUF_NON_COMV: + return &inst->buffers.non_comv; + case BUF_LINE: + return &inst->buffers.line; + case BUF_DPB: + return &inst->buffers.dpb; + case BUF_PERSIST: + return &inst->buffers.persist; + case BUF_VPSS: + return &inst->buffers.vpss; + default: + return NULL; + } +} + +int iris_allocate_buffers(struct iris_inst *inst, + enum iris_buffer_type buf_type, + u32 num_buffers) +{ + struct iris_buffer *buf = NULL; + struct iris_buffers *buffers; + int idx = 0; + + buffers = iris_get_buffer_list(inst, buf_type); + if (!buffers) + return -EINVAL; + + for (idx = 0; idx < num_buffers; idx++) { + buf = kzalloc(sizeof(*buf), GFP_KERNEL); + if (!buf) + return -EINVAL; + + INIT_LIST_HEAD(&buf->list); + list_add_tail(&buf->list, &buffers->list); + buf->type = buf_type; + buf->index = idx; + } + + return 0; +} + +int iris_free_buffers(struct iris_inst *inst, + enum iris_buffer_type buf_type) +{ + struct iris_buffer *buf, *dummy; + struct iris_buffers *buffers; + + buffers = iris_get_buffer_list(inst, buf_type); + if (!buffers) + return -EINVAL; + + list_for_each_entry_safe(buf, dummy, &buffers->list, list) { + list_del_init(&buf->list); + kfree(buf); + } + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h new file mode 100644 index 0000000..1cd76a9 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h @@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_BUFFER_H_ +#define _IRIS_BUFFER_H_ + +#define MIN_BUFFERS 4 + +#include "iris_common.h" + +struct iris_inst; + +struct iris_buffers { + struct list_head list; // list of "struct iris_buffer" + u32 min_count; + u32 actual_count; + u32 size; + bool reuse; +}; + +struct iris_buffers_info { + struct iris_buffers input; + struct iris_buffers output; + struct iris_buffers read_only; + struct iris_buffers bin; + struct iris_buffers arp; + struct iris_buffers comv; + struct iris_buffers non_comv; + struct iris_buffers line; + struct iris_buffers dpb; + struct iris_buffers persist; + struct iris_buffers vpss; +}; + +int iris_get_buf_min_count(struct iris_inst *inst, + enum iris_buffer_type buffer_type); +int iris_get_buffer_size(struct iris_inst *inst, + enum iris_buffer_type buffer_type); +struct iris_buffers *iris_get_buffer_list(struct iris_inst *inst, + enum iris_buffer_type buffer_type); +int iris_allocate_buffers(struct iris_inst *inst, + enum iris_buffer_type buf_type, + u32 num_buffers); +int iris_free_buffers(struct iris_inst *inst, + enum iris_buffer_type buf_type); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h index 3e4dd71..4edadc3 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -5,6 +5,7 @@ #ifndef _IRIS_COMMON_H_ #define _IRIS_COMMON_H_ +#include #include #include #include @@ -14,7 +15,31 @@ #define OUTPUT_MPLANE V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE #define DEFAULT_WIDTH 320 #define DEFAULT_HEIGHT 240 -#define DEFAULT_BUF_SIZE 115200 + +#define MB_IN_PIXEL (16 * 16) + +#define NUM_MBS_4k (((4096 + 15) >> 4) * ((2304 + 15) >> 4)) + +enum codec_type { + H264 = BIT(0), + HEVC = BIT(1), + VP9 = BIT(2), +}; + +enum colorformat_type { + FMT_NONE = 0, + FMT_NV12C = BIT(0), + FMT_NV12 = BIT(1), + FMT_NV21 = BIT(2), + FMT_TP10C = BIT(3), +}; + +struct rect_desc { + u32 left; + u32 top; + u32 width; + u32 height; +}; enum signal_session_response { SIGNAL_CMD_STOP_INPUT = 0, @@ -23,4 +48,48 @@ enum signal_session_response { MAX_SIGNAL, }; +enum iris_buffer_type { + BUF_NONE, + BUF_INPUT, + BUF_OUTPUT, + BUF_READ_ONLY, + BUF_BIN, + BUF_ARP, + BUF_COMV, + BUF_NON_COMV, + BUF_LINE, + BUF_DPB, + BUF_PERSIST, + BUF_VPSS, +}; + +enum iris_buffer_attributes { + BUF_ATTR_DEFERRED = BIT(0), + BUF_ATTR_READ_ONLY = BIT(1), + BUF_ATTR_PENDING_RELEASE = BIT(2), + BUF_ATTR_QUEUED = BIT(3), + BUF_ATTR_DEQUEUED = BIT(4), + BUF_ATTR_BUFFER_DONE = BIT(5), +}; + +struct iris_buffer { + struct list_head list; + struct iris_inst *inst; + enum iris_buffer_type type; + u32 index; + int fd; + u32 buffer_size; + u32 data_offset; + u32 data_size; + u64 device_addr; + void *kvaddr; + unsigned long dma_attrs; + u32 flags; + u64 timestamp; + enum iris_buffer_attributes attr; + void *dmabuf; + struct sg_table *sg_table; + struct dma_buf_attachment *attach; +}; + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index 8d8bc3a..54a7851 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -42,6 +42,30 @@ u32 get_port_info(struct iris_inst *inst, return HFI_PORT_NONE; } +enum iris_buffer_type v4l2_type_to_driver(u32 type) +{ + switch (type) { + case INPUT_MPLANE: + return BUF_INPUT; + case OUTPUT_MPLANE: + return BUF_OUTPUT; + default: + return 0; + } +} + +int get_mbpf(struct iris_inst *inst) +{ + int height = 0, width = 0; + struct v4l2_format *inp_f; + + inp_f = inst->fmt_src; + width = max(inp_f->fmt.pix_mp.width, inst->crop.width); + height = max(inp_f->fmt.pix_mp.height, inst->crop.height); + + return NUM_MBS_PER_FRAME(height, width); +} + static int process_inst_timeout(struct iris_inst *inst) { struct iris_inst *instance; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index 60c79124..3bae969 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -11,6 +11,7 @@ #include "iris_core.h" #include "iris_instance.h" +#include "iris_buffer.h" #include "platform_common.h" #define NUM_MBS_PER_FRAME(__height, __width) \ @@ -21,7 +22,8 @@ bool res_is_less_than(u32 width, u32 height, u32 ref_width, u32 ref_height); u32 get_port_info(struct iris_inst *inst, enum plat_inst_cap_type cap_id); - +enum iris_buffer_type v4l2_type_to_driver(u32 type); +int get_mbpf(struct iris_inst *inst); int close_session(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h index f6a3066..a5c6cb48 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -8,6 +8,8 @@ #include +#include "iris_buffer.h" +#include "iris_common.h" #include "iris_core.h" #include "iris_common.h" #include "platform_common.h" @@ -25,6 +27,7 @@ * @fmt_src: structure of v4l2_format for source * @fmt_dst: structure of v4l2_format for destination * @ctrl_handler: reference of v4l2 ctrl handler + * @crop: structure of crop info * @packet: HFI packet * @packet_size: HFI packet size * @completions: structure of signal completions @@ -32,6 +35,7 @@ * @num_ctrls: supported number of controls * @caps_list: list head of capability * @codec: codec type + * @buffers: structure of buffer info */ struct iris_inst { @@ -45,6 +49,7 @@ struct iris_inst { struct v4l2_format *fmt_src; struct v4l2_format *fmt_dst; struct v4l2_ctrl_handler ctrl_handler; + struct rect_desc crop; void *packet; u32 packet_size; struct completion completions[MAX_SIGNAL]; @@ -52,6 +57,7 @@ struct iris_inst { u32 num_ctrls; struct list_head caps_list; enum codec_type codec; + struct iris_buffers_info buffers; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c new file mode 100644 index 0000000..b040d27 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_buffer.h" +#include "iris_core.h" +#include "iris_helpers.h" +#include "iris_instance.h" +#include "iris_vb2.h" + +int iris_vb2_queue_setup(struct vb2_queue *q, + unsigned int *num_buffers, unsigned int *num_planes, + unsigned int sizes[], struct device *alloc_devs[]) +{ + enum iris_buffer_type buffer_type = 0; + struct iris_buffers *buffers; + struct iris_inst *inst; + struct iris_core *core; + struct v4l2_format *f; + int ret; + + if (!q || !num_buffers || !num_planes || !sizes) + return -EINVAL; + + inst = vb2_get_drv_priv(q); + if (!inst || !inst->core) + return -EINVAL; + + core = inst->core; + + if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) + f = inst->fmt_src; + else + f = inst->fmt_dst; + + if (*num_planes) { + if (*num_planes != f->fmt.pix_mp.num_planes || + sizes[0] < f->fmt.pix_mp.plane_fmt[0].sizeimage) + return -EINVAL; + } + + buffer_type = v4l2_type_to_driver(q->type); + if (!buffer_type) + return -EINVAL; + + ret = iris_free_buffers(inst, buffer_type); + if (ret) + return ret; + + buffers = iris_get_buffer_list(inst, buffer_type); + if (!buffers) + return -EINVAL; + + buffers->min_count = iris_get_buf_min_count(inst, buffer_type); + if (*num_buffers < buffers->min_count) + *num_buffers = buffers->min_count; + buffers->actual_count = *num_buffers; + *num_planes = 1; + + buffers->size = iris_get_buffer_size(inst, buffer_type); + + f->fmt.pix_mp.plane_fmt[0].sizeimage = buffers->size; + sizes[0] = f->fmt.pix_mp.plane_fmt[0].sizeimage; + + ret = iris_allocate_buffers(inst, buffer_type, *num_buffers); + + q->dev = core->dev; + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h new file mode 100644 index 0000000..8a8e1039 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_VB2_H_ +#define _IRIS_VB2_H_ + +#include + +int iris_vb2_queue_setup(struct vb2_queue *q, + unsigned int *num_buffers, unsigned int *num_planes, + unsigned int sizes[], struct device *alloc_devs[]); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index 984be34..1b957a13 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -3,6 +3,7 @@ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include "iris_buffer.h" #include "iris_common.h" #include "iris_vdec.h" @@ -22,7 +23,7 @@ void vdec_inst_init(struct iris_inst *inst) f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_H264; f->fmt.pix_mp.num_planes = 1; f->fmt.pix_mp.plane_fmt[0].bytesperline = 0; - f->fmt.pix_mp.plane_fmt[0].sizeimage = DEFAULT_BUF_SIZE; + f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_INPUT); f->fmt.pix_mp.field = V4L2_FIELD_NONE; f = inst->fmt_dst; @@ -32,7 +33,7 @@ void vdec_inst_init(struct iris_inst *inst) f->fmt.pix_mp.height = ALIGN(DEFAULT_HEIGHT, 32); f->fmt.pix_mp.num_planes = 1; f->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(DEFAULT_WIDTH, 128); - f->fmt.pix_mp.plane_fmt[0].sizeimage = DEFAULT_BUF_SIZE; + f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT); f->fmt.pix_mp.field = V4L2_FIELD_NONE; f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c index 3a26edb..410de720 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -10,6 +10,7 @@ #include "iris_vdec.h" #include "iris_vidc.h" #include "iris_ctrls.h" +#include "iris_vb2.h" static int vidc_v4l2_fh_init(struct iris_inst *inst) { @@ -168,13 +169,22 @@ int vidc_open(struct file *filp) inst->core = core; inst->session_id = hash32_ptr(inst); mutex_init(&inst->ctx_q_lock); - for (i = 0; i < MAX_SIGNAL; i++) - init_completion(&inst->completions[i]); ret = vidc_add_session(inst); if (ret) goto fail_free_inst; + INIT_LIST_HEAD(&inst->buffers.input.list); + INIT_LIST_HEAD(&inst->buffers.output.list); + INIT_LIST_HEAD(&inst->buffers.read_only.list); + INIT_LIST_HEAD(&inst->buffers.bin.list); + INIT_LIST_HEAD(&inst->buffers.arp.list); + INIT_LIST_HEAD(&inst->buffers.comv.list); + INIT_LIST_HEAD(&inst->buffers.non_comv.list); + INIT_LIST_HEAD(&inst->buffers.line.list); + INIT_LIST_HEAD(&inst->buffers.dpb.list); + INIT_LIST_HEAD(&inst->buffers.persist.list); + INIT_LIST_HEAD(&inst->buffers.vpss.list); INIT_LIST_HEAD(&inst->caps_list); for (i = 0; i < MAX_SIGNAL; i++) init_completion(&inst->completions[i]); @@ -306,9 +316,14 @@ static const struct v4l2_file_operations v4l2_file_ops = { .poll = vidc_poll, }; +static const struct vb2_ops iris_vb2_ops = { + .queue_setup = iris_vb2_queue_setup, +}; + int init_ops(struct iris_core *core) { core->v4l2_file_ops = &v4l2_file_ops; + core->vb2_ops = &iris_vb2_ops; return 0; } diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h index 8305c65..e242614 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h @@ -9,6 +9,8 @@ #include #include +#include "iris_common.h" + struct iris_core; struct iris_inst; @@ -35,20 +37,6 @@ struct iris_inst; .bank_spreading = bsp, \ } -enum codec_type { - H264 = BIT(0), - HEVC = BIT(1), - VP9 = BIT(2), -}; - -enum colorformat_type { - FMT_NONE = 0, - FMT_NV12C = BIT(0), - FMT_NV12 = BIT(1), - FMT_NV21 = BIT(2), - FMT_TP10C = BIT(3), -}; - enum stage_type { STAGE_NONE = 0, STAGE_1 = 1, From patchwork Mon Dec 18 11:32:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755791 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DDB14200C7; Mon, 18 Dec 2023 11:40:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="Iow/hLS0" Received: from pps.filterd (m0279865.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIBDuCL018171; Mon, 18 Dec 2023 11:40:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=U89F2RsbUBam7okcX4qquI23KYt3c/bBrBXYoIhu7nc=; b=Io w/hLS0iYU+2wA536pvvoU0BPIdStACjIV8AU/h+87eQsegFLHRhM6Hxqxh8Lmk3B SSFddYPVaBBwJ0cxjOS84ggBpae47gPxsAQ2qQWADmiKqyDuUAje1ccXKEz1f6Gn uifQHzkc7MRQJAhUKCWtcTB00nJ6dYqB1XVE7evRhqKmkD0RLFLX86S6PkAQltAO M5TFvN1uezVsptxx+7JZem5mEJjSLJVboayoPe9qTT+seWeTBEbrPxd35dND7KPF D0eYtfrnDmEQNvERMLgnQdxpSN4z1yuKdgUR90vkfHQyuHSQnHnJXjlX+6MDa2KM AaFvGrcc/izLM855ieWQ== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2n1781tu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:03 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBe0wx004854; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ag-3; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX8Ya030075; Mon, 18 Dec 2023 11:33:08 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX8hq030062; Mon, 18 Dec 2023 11:33:08 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 4C494230A; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 18/34] media: iris: introduce and implement iris vb2 mem ops Date: Mon, 18 Dec 2023 17:02:13 +0530 Message-Id: <1702899149-21321-19-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: X0VFrL2CzS5cfywwCIMg4PGknbvtbMVt X-Proofpoint-GUID: X0VFrL2CzS5cfywwCIMg4PGknbvtbMVt X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=985 mlxscore=0 lowpriorityscore=0 impostorscore=0 bulkscore=0 adultscore=0 clxscore=1015 priorityscore=1501 suspectscore=0 malwarescore=0 phishscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Vikash Garodia Implement the iris vb2 mem ops for buffer management for DMABUF streaming mode. Update video driver buffer with dma buf information. Signed-off-by: Vikash Garodia Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/iris_probe.c | 1 + drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 147 +++++++++++++++++++++ drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 10 ++ .../media/platform/qcom/vcodec/iris/iris_vidc.c | 11 ++ 4 files changed, 169 insertions(+) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index 50fb93e..bf484a3 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -233,5 +233,6 @@ static struct platform_driver qcom_iris_driver = { }; module_platform_driver(qcom_iris_driver); +MODULE_IMPORT_NS(DMA_BUF); MODULE_DESCRIPTION("Qualcomm Iris video driver"); MODULE_LICENSE("GPL"); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c index b040d27..a57b5fb 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c @@ -69,3 +69,150 @@ int iris_vb2_queue_setup(struct vb2_queue *q, return ret; } + +void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev, + struct dma_buf *dbuf, unsigned long size) +{ + enum iris_buffer_type buf_type; + struct iris_buffers *buffers; + struct iris_buffer *iter; + struct iris_buffer *buf; + struct iris_inst *inst; + bool found = false; + + if (!vb || !dev || !dbuf || !vb->vb2_queue) + return ERR_PTR(-EINVAL); + + inst = vb->vb2_queue->drv_priv; + + buf_type = v4l2_type_to_driver(vb->type); + + buffers = iris_get_buffer_list(inst, buf_type); + if (!buffers) + return NULL; + + list_for_each_entry(iter, &buffers->list, list) { + if (iter->index == vb->index) { + found = true; + buf = iter; + break; + } + } + + if (!found) + return NULL; + + buf->inst = inst; + buf->dmabuf = dbuf; + + buf->attach = dma_buf_attach(dbuf, dev); + if (IS_ERR(buf->attach)) { + buf->attach = NULL; + return NULL; + } + + return buf; +} + +int iris_vb2_map_dmabuf(void *buf_priv) +{ + struct iris_buffer *buf = buf_priv; + struct iris_core *core; + struct iris_inst *inst; + + if (!buf || !buf->inst) + return -EINVAL; + + inst = buf->inst; + core = inst->core; + + if (!buf->attach) { + dev_err(core->dev, "trying to map a non attached buffer\n"); + return -EINVAL; + } + + buf->sg_table = dma_buf_map_attachment(buf->attach, DMA_BIDIRECTIONAL); + if (IS_ERR(buf->sg_table)) + return -EINVAL; + + if (!buf->sg_table->sgl) { + dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL); + buf->sg_table = NULL; + return -EINVAL; + } + + buf->device_addr = sg_dma_address(buf->sg_table->sgl); + + return 0; +} + +void iris_vb2_unmap_dmabuf(void *buf_priv) +{ + struct iris_buffer *buf = buf_priv; + struct iris_core *core; + struct iris_inst *inst; + + if (!buf || !buf->inst) + return; + + inst = buf->inst; + core = inst->core; + + if (!buf->attach) { + dev_err(core->dev, "trying to unmap a non attached buffer\n"); + return; + } + + if (!buf->sg_table) { + dev_err(core->dev, "dmabuf buffer is already unmapped\n"); + return; + } + + if (buf->attach && buf->sg_table) { + dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL); + buf->sg_table = NULL; + buf->device_addr = 0x0; + } +} + +void iris_vb2_detach_dmabuf(void *buf_priv) +{ + struct iris_buffer *buf = buf_priv; + struct iris_core *core; + struct iris_inst *inst; + + if (!buf || !buf->inst) + return; + + inst = buf->inst; + core = inst->core; + + if (buf->sg_table) { + dev_err(core->dev, "trying to detach an unmapped buffer\n"); + dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL); + buf->sg_table = NULL; + } + + if (buf->attach && buf->dmabuf) { + dma_buf_detach(buf->dmabuf, buf->attach); + buf->attach = NULL; + } + + buf->dmabuf = NULL; + buf->inst = NULL; +} + +void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev, + unsigned long size) +{ + return (void *)0xdeadbeef; +} + +void iris_vb2_put(void *buf_priv) +{ +} + +int iris_vb2_mmap(void *buf_priv, struct vm_area_struct *vma) +{ + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h index 8a8e1039..4342034 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h @@ -12,4 +12,14 @@ int iris_vb2_queue_setup(struct vb2_queue *q, unsigned int *num_buffers, unsigned int *num_planes, unsigned int sizes[], struct device *alloc_devs[]); +/* vb2_mem_ops */ +void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev, unsigned long size); +void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev, struct dma_buf *dbuf, + unsigned long size); +void iris_vb2_put(void *buf_priv); +int iris_vb2_mmap(void *buf_priv, struct vm_area_struct *vma); +void iris_vb2_detach_dmabuf(void *buf_priv); +int iris_vb2_map_dmabuf(void *buf_priv); +void iris_vb2_unmap_dmabuf(void *buf_priv); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c index 410de720..124333a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -320,10 +320,21 @@ static const struct vb2_ops iris_vb2_ops = { .queue_setup = iris_vb2_queue_setup, }; +static struct vb2_mem_ops iris_vb2_mem_ops = { + .alloc = iris_vb2_alloc, + .put = iris_vb2_put, + .mmap = iris_vb2_mmap, + .attach_dmabuf = iris_vb2_attach_dmabuf, + .detach_dmabuf = iris_vb2_detach_dmabuf, + .map_dmabuf = iris_vb2_map_dmabuf, + .unmap_dmabuf = iris_vb2_unmap_dmabuf, +}; + int init_ops(struct iris_core *core) { core->v4l2_file_ops = &v4l2_file_ops; core->vb2_ops = &iris_vb2_ops; + core->vb2_mem_ops = &iris_vb2_mem_ops; return 0; } From patchwork Mon Dec 18 11:32:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755787 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 54D243B2A8; Mon, 18 Dec 2023 11:40:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="HPHbcOzD" Received: from pps.filterd (m0279867.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIArrqu023249; Mon, 18 Dec 2023 11:40:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=Z+YyFcHZoap2j/dtnB//4TT29/XubabIqnFS3x68d88=; b=HP HbcOzDWp7UlWaoaKcCuEb5T6KOP4q547IkWThF+crClrpXawwacmMldnptjDN5Jr S+Et8v2377q0AaeIsP/KfecRZYgilMuOyGGibOxwzn+sb9IKHu0uIvk14d/HBxgr 9ctOiYgaQTkhSKLDH76DAjvuNZ1jzn/vKeQGhpzvIro2av33oYMbn/Nnl3zKNbfX xelRpPnWy4vBZ0LVWGIEe/QcqS/l7LMfcICoqc+/946s0zkZGroi38chvPF0PFvu tT3GQg+rbhOdEYXHfdfDh21swvz2qP+Z7vNI7Qg6NBlOAnrQ+JBff4I0/xECw3u4 CTDmaSN/nbVqqvWcpOsA== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2md2r4n7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:04 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBe0Ho004853; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ae-4; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7dP029984; Mon, 18 Dec 2023 11:36:07 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6a8029970; Mon, 18 Dec 2023 11:36:07 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 51204230C; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 20/34] media: iris: add video hardware internal buffer count and size calculation Date: Mon, 18 Dec 2023 17:02:15 +0530 Message-Id: <1702899149-21321-21-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: 0cYxUNueEMdBjUclxbGTb8vDKFBWxrpS X-Proofpoint-GUID: 0cYxUNueEMdBjUclxbGTb8vDKFBWxrpS X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_02,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 priorityscore=1501 malwarescore=0 adultscore=0 phishscore=0 bulkscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 suspectscore=0 lowpriorityscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Video driver needs various kind of internal buffers for frame processing. Internal buffer size calculation depends on hardware architecture, color format, resolution and codec. Add APIs to calculate min count and size of different internal buffers for different codecs for iris3. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 1 + .../media/platform/qcom/vcodec/iris/iris_buffer.c | 48 ++ .../media/platform/qcom/vcodec/iris/iris_common.h | 1 + .../media/platform/qcom/vcodec/iris/iris_core.h | 2 + .../media/platform/qcom/vcodec/iris/iris_helpers.c | 13 + .../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 + .../platform/qcom/vcodec/iris/iris_instance.h | 2 + .../media/platform/qcom/vcodec/iris/iris_vdec.c | 1 + .../media/platform/qcom/vcodec/iris/vpu_common.h | 8 + .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 6 + .../platform/qcom/vcodec/iris/vpu_iris3_buffer.c | 201 +++++ .../platform/qcom/vcodec/iris/vpu_iris3_buffer.h | 845 +++++++++++++++++++++ 12 files changed, 1132 insertions(+) create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index a94e36b..7e3d9f1 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -16,6 +16,7 @@ iris-objs += iris_probe.o \ resources.o \ vpu_common.o \ vpu_iris3.o \ + vpu_iris3_buffer.o \ platform_common.o \ platform_sm8550.o diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c index b9cffbf..6d4e722 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.c @@ -17,6 +17,14 @@ static int output_min_count(struct iris_inst *inst) { int output_min_count; + /* fw_min_count > 0 indicates reconfig event has already arrived */ + if (inst->fw_min_count) { + if (is_split_mode_enabled(inst) && inst->codec == VP9) + return min_t(u32, 4, inst->fw_min_count); + else + return inst->fw_min_count; + } + switch (inst->codec) { case H264: case HEVC: @@ -33,6 +41,38 @@ static int output_min_count(struct iris_inst *inst) return output_min_count; } +static u32 internal_buffer_count(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + u32 count = 0; + + if (buffer_type == BUF_BIN || buffer_type == BUF_LINE || + buffer_type == BUF_PERSIST) { + count = 1; + } else if (buffer_type == BUF_COMV || buffer_type == BUF_NON_COMV) { + if (inst->codec == H264 || inst->codec == HEVC) + count = 1; + else + count = 0; + } else { + count = 0; + } + + return count; +} + +static int dpb_count(struct iris_inst *inst) +{ + int count = 0; + + if (is_split_mode_enabled(inst)) { + count = inst->fw_min_count ? + inst->fw_min_count : inst->buffers.output.min_count; + } + + return count; +} + int iris_get_buf_min_count(struct iris_inst *inst, enum iris_buffer_type buffer_type) { @@ -41,6 +81,14 @@ int iris_get_buf_min_count(struct iris_inst *inst, return input_min_count(inst); case BUF_OUTPUT: return output_min_count(inst); + case BUF_BIN: + case BUF_COMV: + case BUF_NON_COMV: + case BUF_LINE: + case BUF_PERSIST: + return internal_buffer_count(inst, buffer_type); + case BUF_DPB: + return dpb_count(inst); default: return 0; } diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h index 4edadc3..a83d1c1 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -15,6 +15,7 @@ #define OUTPUT_MPLANE V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE #define DEFAULT_WIDTH 320 #define DEFAULT_HEIGHT 240 +#define DEFAULT_BSE_VPP_DELAY 2 #define MB_IN_PIXEL (16 * 16) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index 30f7ad7..154991c 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -50,6 +50,7 @@ * @header_id: id of packet header * @packet_id: id of packet * @vpu_ops: a pointer to vpu ops + * @session_ops: a pointer to session level ops * @dec_codecs_count: supported codec count for decoder * @platform_data: a structure for platform data * @cap: an array for supported core capabilities @@ -91,6 +92,7 @@ struct iris_core { u32 header_id; u32 packet_id; const struct vpu_ops *vpu_ops; + const struct vpu_session_ops *session_ops; u32 dec_codecs_count; struct platform_data *platform_data; struct plat_core_cap cap[CORE_CAP_MAX + 1]; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index 54a7851..335885f 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -66,6 +66,19 @@ int get_mbpf(struct iris_inst *inst) return NUM_MBS_PER_FRAME(height, width); } +bool is_linear_colorformat(u32 colorformat) +{ + return colorformat == V4L2_PIX_FMT_NV12 || colorformat == V4L2_PIX_FMT_NV21; +} + +bool is_split_mode_enabled(struct iris_inst *inst) +{ + if (is_linear_colorformat(inst->fmt_dst->fmt.pix_mp.pixelformat)) + return true; + + return false; +} + static int process_inst_timeout(struct iris_inst *inst) { struct iris_inst *instance; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index 3bae969..9e85510 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -12,6 +12,7 @@ #include "iris_core.h" #include "iris_instance.h" #include "iris_buffer.h" +#include "iris_instance.h" #include "platform_common.h" #define NUM_MBS_PER_FRAME(__height, __width) \ @@ -26,4 +27,7 @@ enum iris_buffer_type v4l2_type_to_driver(u32 type); int get_mbpf(struct iris_inst *inst); int close_session(struct iris_inst *inst); +bool is_linear_colorformat(u32 colorformat); +bool is_split_mode_enabled(struct iris_inst *inst); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h index a5c6cb48..5d4c856 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -36,6 +36,7 @@ * @caps_list: list head of capability * @codec: codec type * @buffers: structure of buffer info + * @fw_min_count: minimnum count of buffers needed by fw */ struct iris_inst { @@ -58,6 +59,7 @@ struct iris_inst { struct list_head caps_list; enum codec_type codec; struct iris_buffers_info buffers; + u32 fw_min_count; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index 1b957a13..b131a50 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -39,6 +39,7 @@ void vdec_inst_init(struct iris_inst *inst) f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; + inst->fw_min_count = 0; } void vdec_inst_deinit(struct iris_inst *inst) diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h index 6512039..7fba8c2 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h @@ -26,6 +26,14 @@ struct vpu_ops { int (*watchdog)(struct iris_core *core, u32 intr_status); }; +#define call_session_op(c, op, ...) \ + (((c) && (c)->session_ops && (c)->session_ops->op) ? \ + ((c)->session_ops->op(__VA_ARGS__)) : 0) + +struct vpu_session_ops { + int (*int_buf_size)(struct iris_inst *inst, enum iris_buffer_type type); +}; + int init_vpu(struct iris_core *core); int write_register(struct iris_core *core, u32 reg, u32 value); diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c index a34d0ed..efea5aa 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c @@ -6,6 +6,7 @@ #include #include "vpu_iris3.h" +#include "vpu_iris3_buffer.h" #define VIDEO_ARCH_LX 1 @@ -202,9 +203,14 @@ static const struct vpu_ops iris3_ops = { .watchdog = watchdog_iris3, }; +static const struct vpu_session_ops iris3_session_ops = { + .int_buf_size = iris_int_buf_size_iris3, +}; + int init_iris3(struct iris_core *core) { core->vpu_ops = &iris3_ops; + core->session_ops = &iris3_session_ops; return 0; } diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c new file mode 100644 index 0000000..44f9342 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.c @@ -0,0 +1,201 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_core.h" +#include "iris_helpers.h" +#include "iris_instance.h" +#include "vpu_iris3_buffer.h" + +static u32 dec_bin_size_iris3(struct iris_inst *inst) +{ + u32 width, height, num_vpp_pipes; + struct iris_core *core; + struct v4l2_format *f; + u32 size = 0; + + core = inst->core; + + num_vpp_pipes = core->cap[NUM_VPP_PIPE].value; + + f = inst->fmt_src; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + if (inst->codec == H264) + size = hfi_buffer_bin_h264d(width, height, num_vpp_pipes); + else if (inst->codec == HEVC) + size = hfi_buffer_bin_h265d(width, height, num_vpp_pipes); + else if (inst->codec == VP9) + size = hfi_buffer_bin_vp9d(width, height, + num_vpp_pipes); + return size; +} + +static u32 dec_comv_size_iris3(struct iris_inst *inst) +{ + u32 width, height, num_comv; + struct v4l2_format *f; + u32 size = 0; + + f = inst->fmt_src; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + num_comv = inst->buffers.output.min_count; + + if (inst->codec == H264) + size = hfi_buffer_comv_h264d(width, height, num_comv); + else if (inst->codec == HEVC) + size = hfi_buffer_comv_h265d(width, height, num_comv); + + inst->cap[NUM_COMV].value = num_comv; + + return size; +} + +static u32 dec_non_comv_size_iris3(struct iris_inst *inst) +{ + u32 width, height, num_vpp_pipes; + struct iris_core *core; + struct v4l2_format *f; + u32 size = 0; + + core = inst->core; + + num_vpp_pipes = core->cap[NUM_VPP_PIPE].value; + + f = inst->fmt_src; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + if (inst->codec == H264) + size = hfi_buffer_non_comv_h264d(width, height, num_vpp_pipes); + else if (inst->codec == HEVC) + size = hfi_buffer_non_comv_h265d(width, height, num_vpp_pipes); + + return size; +} + +static u32 dec_line_size_iris3(struct iris_inst *inst) +{ + u32 width, height, out_min_count, num_vpp_pipes; + struct iris_core *core; + struct v4l2_format *f; + u32 size = 0; + bool is_opb; + + core = inst->core; + num_vpp_pipes = core->cap[NUM_VPP_PIPE].value; + + is_opb = true; + + f = inst->fmt_src; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + out_min_count = inst->buffers.output.min_count; + if (inst->codec == H264) + size = hfi_buffer_line_h264d(width, height, is_opb, + num_vpp_pipes); + else if (inst->codec == HEVC) + size = hfi_buffer_line_h265d(width, height, is_opb, + num_vpp_pipes); + else if (inst->codec == VP9) + size = hfi_buffer_line_vp9d(width, height, out_min_count, + is_opb, num_vpp_pipes); + return size; +} + +static u32 dec_persist_size_iris3(struct iris_inst *inst) +{ + u32 size = 0; + + if (inst->codec == H264) + size = hfi_buffer_persist_h264d(0); + else if (inst->codec == HEVC) + size = hfi_buffer_persist_h265d(0); + else if (inst->codec == VP9) + size = hfi_buffer_persist_vp9d(); + + return size; +} + +static u32 dec_dpb_size_iris3(struct iris_inst *inst) +{ + struct v4l2_format *f; + u32 width, height; + u32 color_fmt; + u32 size = 0; + + f = inst->fmt_dst; + color_fmt = f->fmt.pix_mp.pixelformat; + if (!is_linear_colorformat(color_fmt)) + return size; + + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + if (color_fmt == V4L2_PIX_FMT_NV12 || + color_fmt == V4L2_PIX_FMT_QC08C) { + size = + hfi_nv12_ubwc_il_calc_buf_size_v2(width, height, + ALIGN(width, 128), + ALIGN(height, 32), + ALIGN(width, 128), + ALIGN((height + 1) >> 1, 32), + ALIGN(DIV_ROUND_UP(width, 32), 64), + ALIGN(DIV_ROUND_UP(height, 8), 16), + ALIGN(DIV_ROUND_UP((width + 1) >> 1, 16), 64), + ALIGN(DIV_ROUND_UP((height + 1) >> 1, 8), 16)); + } else if (color_fmt == V4L2_PIX_FMT_QC10C) { + size = + hfi_yuv420_tp10_ubwc_calc_buf_size(ALIGN(ALIGN(width, 192) * 4 / 3, 256), + ALIGN(height, 16), + ALIGN(ALIGN(width, 192) * 4 / 3, 256), + ALIGN((height + 1) >> 1, 16), + ALIGN(DIV_ROUND_UP(width, 48), 64), + ALIGN(DIV_ROUND_UP(height, 4), 16), + ALIGN(DIV_ROUND_UP((width + 1) >> 1, 24), 64), + ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16)); + } + + return size; +} + +struct iris_buf_type_handle { + enum iris_buffer_type type; + u32 (*handle)(struct iris_inst *inst); +}; + +int iris_int_buf_size_iris3(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + const struct iris_buf_type_handle *buf_type_handle_arr = NULL; + u32 size = 0, buf_type_handle_size = 0; + int i; + + static const struct iris_buf_type_handle dec_internal_buf_type_handle[] = { + {BUF_BIN, dec_bin_size_iris3 }, + {BUF_COMV, dec_comv_size_iris3 }, + {BUF_NON_COMV, dec_non_comv_size_iris3 }, + {BUF_LINE, dec_line_size_iris3 }, + {BUF_PERSIST, dec_persist_size_iris3 }, + {BUF_DPB, dec_dpb_size_iris3 }, + }; + + buf_type_handle_size = ARRAY_SIZE(dec_internal_buf_type_handle); + buf_type_handle_arr = dec_internal_buf_type_handle; + + if (!buf_type_handle_arr || !buf_type_handle_size) + return size; + + for (i = 0; i < buf_type_handle_size; i++) { + if (buf_type_handle_arr[i].type == buffer_type) { + size = buf_type_handle_arr[i].handle(inst); + break; + } + } + + return size; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h new file mode 100644 index 0000000..b520c79 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_buffer.h @@ -0,0 +1,845 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _VPU_IRIS3_BUFFER_H_ +#define _VPU_IRIS3_BUFFER_H_ + +#include +#include +#include + +#include "iris_instance.h" + +#define DMA_ALIGNMENT 256 + +#define BUFFER_ALIGNMENT_512_BYTES 512 +#define BUFFER_ALIGNMENT_256_BYTES 256 +#define BUFFER_ALIGNMENT_64_BYTES 64 +#define BUFFER_ALIGNMENT_32_BYTES 32 +#define BUFFER_ALIGNMENT_16_BYTES 16 + +#define HFI_ALIGNMENT_4096 (4096) + +#define HFI_COL_FMT_NV12C_Y_TILE_HEIGHT (8) +#define HFI_COL_FMT_NV12C_Y_TILE_WIDTH (32) +#define HFI_COL_FMT_NV12C_UV_TILE_HEIGHT (8) +#define HFI_COL_FMT_NV12C_UV_TILE_WIDTH (16) + +#define NUM_HW_PIC_BUF 32 +#define SIZE_HW_PIC(size_per_buf) (NUM_HW_PIC_BUF * (size_per_buf)) + +#define MAX_TILE_COLUMNS 32 + +#define LCU_MAX_SIZE_PELS 64 +#define LCU_MIN_SIZE_PELS 16 + +#define HDR10_HIST_EXTRADATA_SIZE (4 * 1024) + +#define BIN_BUFFER_THRESHOLD (1280 * 736) + +#define VPP_CMD_MAX_SIZE (BIT(20)) + +#define H264D_MAX_SLICE 1800 + +#define SIZE_H264D_BUFTAB_T (256) +#define SIZE_H264D_HW_PIC_T (BIT(11)) +#define SIZE_H264D_BSE_CMD_PER_BUF (32 * 4) +#define SIZE_H264D_VPP_CMD_PER_BUF (512) + +#define NUM_SLIST_BUF_H264 (256 + 32) +#define SIZE_SLIST_BUF_H264 (512) +#define H264_DISPLAY_BUF_SIZE (3328) +#define H264_NUM_FRM_INFO (66) + +#define H265_NUM_TILE_COL 32 +#define H265_NUM_TILE_ROW 128 +#define H265_NUM_TILE (H265_NUM_TILE_ROW * H265_NUM_TILE_COL + 1) +#define SIZE_H265D_BSE_CMD_PER_BUF (16 * sizeof(u32)) + +#define NUM_SLIST_BUF_H265 (80 + 20) +#define SIZE_SLIST_BUF_H265 (BIT(10)) +#define H265_DISPLAY_BUF_SIZE (3072) +#define H265_NUM_FRM_INFO (48) + +#define VP9_NUM_FRAME_INFO_BUF 32 +#define VP9_NUM_PROBABILITY_TABLE_BUF (VP9_NUM_FRAME_INFO_BUF + 4) +#define VP9_PROB_TABLE_SIZE (3840) +#define VP9_FRAME_INFO_BUF_SIZE (6144) + +#define VP9_UDC_HEADER_BUF_SIZE (3 * 128) +#define MAX_SUPERFRAME_HEADER_LEN (34) +#define CCE_TILE_OFFSET_SIZE ALIGN(32 * 4 * 4, BUFFER_ALIGNMENT_32_BYTES) + +#define SIZE_SEI_USERDATA (4096) +#define SIZE_DOLBY_RPU_METADATA (41 * 1024) + +#define H264_CABAC_HDR_RATIO_HD_TOT 1 +#define H264_CABAC_RES_RATIO_HD_TOT 3 + +#define H265D_MAX_SLICE 1200 +#define SIZE_H265D_HW_PIC_T SIZE_H264D_HW_PIC_T +#define H265_CABAC_HDR_RATIO_HD_TOT 2 +#define H265_CABAC_RES_RATIO_HD_TOT 2 +#define SIZE_H265D_VPP_CMD_PER_BUF (256) + +#define VPX_DECODER_FRAME_CONCURENCY_LVL (2) +#define VPX_DECODER_FRAME_BIN_HDR_BUDGET 1 +#define VPX_DECODER_FRAME_BIN_RES_BUDGET 3 +#define VPX_DECODER_FRAME_BIN_DENOMINATOR 2 + +#define VPX_DECODER_FRAME_BIN_RES_BUDGET_RATIO (3 / 2) + +#define MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE 64 +#define MAX_FE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE 64 +#define MAX_FE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE 64 + +#define MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE (128 / 8) +#define MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE (128 / 8) +#define MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE (128 / 8) + +#define MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE (64 * 2 * 3) +#define MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE 640 + +static inline +u32 size_h264d_hw_bin_buffer(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes) +{ + u32 size_yuv, size_bin_hdr, size_bin_res; + + size_yuv = ((frame_width * frame_height) <= BIN_BUFFER_THRESHOLD) ? + ((BIN_BUFFER_THRESHOLD * 3) >> 1) : + ((frame_width * frame_height * 3) >> 1); + size_bin_hdr = size_yuv * H264_CABAC_HDR_RATIO_HD_TOT; + size_bin_res = size_yuv * H264_CABAC_RES_RATIO_HD_TOT; + size_bin_hdr = ALIGN(size_bin_hdr / num_vpp_pipes, + DMA_ALIGNMENT) * num_vpp_pipes; + size_bin_res = ALIGN(size_bin_res / num_vpp_pipes, + DMA_ALIGNMENT) * num_vpp_pipes; + + return size_bin_hdr + size_bin_res; +} + +static inline +u32 hfi_buffer_bin_h264d(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes) +{ + u32 n_aligned_w, n_aligned_h; + + n_aligned_w = + ALIGN(frame_width, BUFFER_ALIGNMENT_16_BYTES); + n_aligned_h = + ALIGN(frame_height, BUFFER_ALIGNMENT_16_BYTES); + + return size_h264d_hw_bin_buffer(n_aligned_w, n_aligned_h, + num_vpp_pipes); +} + +static inline +u32 size_h265d_hw_bin_buffer(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes) +{ + u32 size_yuv, size_bin_hdr, size_bin_res; + + size_yuv = ((frame_width * frame_height) <= BIN_BUFFER_THRESHOLD) ? + ((BIN_BUFFER_THRESHOLD * 3) >> 1) : + ((frame_width * frame_height * 3) >> 1); + size_bin_hdr = size_yuv * H265_CABAC_HDR_RATIO_HD_TOT; + size_bin_res = size_yuv * H265_CABAC_RES_RATIO_HD_TOT; + size_bin_hdr = ALIGN(size_bin_hdr / num_vpp_pipes, DMA_ALIGNMENT) * + num_vpp_pipes; + size_bin_res = ALIGN(size_bin_res / num_vpp_pipes, DMA_ALIGNMENT) * + num_vpp_pipes; + + return size_bin_hdr + size_bin_res; +} + +static inline +u32 hfi_buffer_bin_h265d(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes) +{ + u32 n_aligned_w, n_aligned_h; + + n_aligned_w = ALIGN(frame_width, BUFFER_ALIGNMENT_16_BYTES); + n_aligned_h = ALIGN(frame_height, BUFFER_ALIGNMENT_16_BYTES); + return size_h265d_hw_bin_buffer(n_aligned_w, n_aligned_h, + num_vpp_pipes); +} + +static inline +u32 hfi_buffer_bin_vp9d(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes) +{ + u32 _size_yuv, _size; + + _size_yuv = ALIGN(frame_width, BUFFER_ALIGNMENT_16_BYTES) * + ALIGN(frame_height, BUFFER_ALIGNMENT_16_BYTES) * 3 / 2; + _size = ALIGN(((max_t(u32, _size_yuv, ((BIN_BUFFER_THRESHOLD * 3) >> 1)) * + VPX_DECODER_FRAME_BIN_HDR_BUDGET / VPX_DECODER_FRAME_BIN_DENOMINATOR * + VPX_DECODER_FRAME_CONCURENCY_LVL) / num_vpp_pipes), + DMA_ALIGNMENT) + + ALIGN(((max_t(u32, _size_yuv, ((BIN_BUFFER_THRESHOLD * 3) >> 1)) * + VPX_DECODER_FRAME_BIN_RES_BUDGET / VPX_DECODER_FRAME_BIN_DENOMINATOR * + VPX_DECODER_FRAME_CONCURENCY_LVL) / num_vpp_pipes), + DMA_ALIGNMENT); + + return _size * num_vpp_pipes; +} + +static inline +u32 hfi_buffer_comv_h264d(u32 frame_width, u32 frame_height, + u32 _comv_bufcount) +{ + u32 frame_width_in_mbs = ((frame_width + 15) >> 4); + u32 frame_height_in_mbs = ((frame_height + 15) >> 4); + u32 col_mv_aligned_width = (frame_width_in_mbs << 7); + u32 col_zero_aligned_width = (frame_width_in_mbs << 2); + u32 col_zero_size = 0, size_colloc = 0; + + col_mv_aligned_width = + ALIGN(col_mv_aligned_width, BUFFER_ALIGNMENT_16_BYTES); + col_zero_aligned_width = + ALIGN(col_zero_aligned_width, BUFFER_ALIGNMENT_16_BYTES); + col_zero_size = col_zero_aligned_width * + ((frame_height_in_mbs + 1) >> 1); + col_zero_size = + ALIGN(col_zero_size, BUFFER_ALIGNMENT_64_BYTES); + col_zero_size <<= 1; + col_zero_size = + ALIGN(col_zero_size, BUFFER_ALIGNMENT_512_BYTES); + size_colloc = col_mv_aligned_width * + ((frame_height_in_mbs + 1) >> 1); + size_colloc = + ALIGN(size_colloc, BUFFER_ALIGNMENT_64_BYTES); + size_colloc <<= 1; + size_colloc = + ALIGN(size_colloc, BUFFER_ALIGNMENT_512_BYTES); + size_colloc += (col_zero_size + SIZE_H264D_BUFTAB_T * 2); + + return (size_colloc * (_comv_bufcount)) + + BUFFER_ALIGNMENT_512_BYTES; +} + +static inline +u32 hfi_buffer_comv_h265d(u32 frame_width, u32 frame_height, + u32 _comv_bufcount) +{ + u32 _size; + + _size = ALIGN(((((frame_width + 15) >> 4) * + ((frame_height + 15) >> 4)) << 8), + BUFFER_ALIGNMENT_512_BYTES); + _size *= _comv_bufcount; + _size += BUFFER_ALIGNMENT_512_BYTES; + + return _size; +} + +static inline +u32 size_h264d_bse_cmd_buf(u32 frame_width, u32 frame_height) +{ + u32 _height = ALIGN(frame_height, + BUFFER_ALIGNMENT_32_BYTES); + return min_t(u32, (((_height + 15) >> 4) * 48), H264D_MAX_SLICE) * + SIZE_H264D_BSE_CMD_PER_BUF; +} + +static inline +u32 size_h264d_vpp_cmd_buf(u32 frame_width, u32 frame_height) +{ + u32 _size, _height; + + _height = ALIGN(frame_height, BUFFER_ALIGNMENT_32_BYTES); + _size = min_t(u32, (((_height + 15) >> 4) * 48), H264D_MAX_SLICE) * + SIZE_H264D_VPP_CMD_PER_BUF; + + if (_size > VPP_CMD_MAX_SIZE) + _size = VPP_CMD_MAX_SIZE; + + return _size; +} + +static inline +u32 hfi_buffer_non_comv_h264d(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes) +{ + u32 _size_bse, _size_vpp, _size; + + _size_bse = size_h264d_bse_cmd_buf(frame_width, frame_height); + _size_vpp = size_h264d_vpp_cmd_buf(frame_width, frame_height); + _size = ALIGN(_size_bse, DMA_ALIGNMENT) + + ALIGN(_size_vpp, DMA_ALIGNMENT) + + ALIGN(SIZE_HW_PIC(SIZE_H264D_HW_PIC_T), DMA_ALIGNMENT); + + return ALIGN(_size, DMA_ALIGNMENT); +} + +static inline +u32 size_h265d_bse_cmd_buf(u32 frame_width, u32 frame_height) +{ + u32 _size; + + _size = ALIGN(((ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS) * + (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS)) * + NUM_HW_PIC_BUF, DMA_ALIGNMENT); + _size = min_t(u32, _size, H265D_MAX_SLICE + 1); + _size = 2 * _size * SIZE_H265D_BSE_CMD_PER_BUF; + + return _size; +} + +static inline +u32 size_h265d_vpp_cmd_buf(u32 frame_width, u32 frame_height) +{ + u32 _size; + + _size = ALIGN(((ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS) * + (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS)) * + NUM_HW_PIC_BUF, DMA_ALIGNMENT); + _size = min_t(u32, _size, H265D_MAX_SLICE + 1); + _size = ALIGN(_size, 4); + _size = 2 * _size * SIZE_H265D_VPP_CMD_PER_BUF; + if (_size > VPP_CMD_MAX_SIZE) + _size = VPP_CMD_MAX_SIZE; + + return _size; +} + +static inline +u32 hfi_buffer_non_comv_h265d(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes) +{ + u32 _size_bse, _size_vpp, _size; + + _size_bse = size_h265d_bse_cmd_buf(frame_width, frame_height); + _size_vpp = size_h265d_vpp_cmd_buf(frame_width, frame_height); + _size = ALIGN(_size_bse, DMA_ALIGNMENT) + + ALIGN(_size_vpp, DMA_ALIGNMENT) + + ALIGN(NUM_HW_PIC_BUF * 20 * 22 * 4, DMA_ALIGNMENT) + + ALIGN(2 * sizeof(u16) * + (ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS) * + (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS), + DMA_ALIGNMENT) + + ALIGN(SIZE_HW_PIC(SIZE_H265D_HW_PIC_T), + DMA_ALIGNMENT) + + HDR10_HIST_EXTRADATA_SIZE; + + return ALIGN(_size, DMA_ALIGNMENT); +} + +static inline u32 size_h264d_lb_fe_top_data(u32 frame_width, u32 frame_height) +{ + return MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE * ALIGN(frame_width, 16) * 3; +} + +static inline u32 size_h264d_lb_fe_top_ctrl(u32 frame_width, u32 frame_height) +{ + return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * ((frame_width + 15) >> 4); +} + +static inline u32 size_h264d_lb_fe_left_ctrl(u32 frame_width, u32 frame_height) +{ + return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * ((frame_height + 15) >> 4); +} + +static inline u32 size_h264d_lb_se_top_ctrl(u32 frame_width, u32 frame_height) +{ + return MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * ((frame_width + 15) >> 4); +} + +static inline u32 size_h264d_lb_se_left_ctrl(u32 frame_width, u32 frame_height) +{ + return MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * ((frame_height + 15) >> 4); +} + +static inline u32 size_h264d_lb_pe_top_data(u32 frame_width, u32 frame_height) +{ + return MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE * ((frame_width + 15) >> 4); +} + +static inline u32 size_h264d_lb_vsp_top(u32 frame_width, u32 frame_height) +{ + return (((frame_width + 15) >> 4) << 7); +} + +static inline u32 size_h264d_lb_recon_dma_metadata_wr(u32 frame_width, u32 frame_height) +{ + return ALIGN(frame_height, 16) * 32; +} + +static inline u32 size_h264d_qp(u32 frame_width, u32 frame_height) +{ + return ((frame_width + 63) >> 6) * ((frame_height + 63) >> 6) * 128; +} + +static inline +u32 size_vpss_lb(u32 frame_width, u32 frame_height, u32 num_vpp_pipes) +{ + u32 vpss_4tap_left_buffer_size = 0, vpss_div2_left_buffer_size = 0; + u32 vpss_4tap_top_buffer_size = 0, vpss_div2_top_buffer_size = 0; + u32 opb_lb_wr_llb_y_buffer_size, opb_lb_wr_llb_uv_buffer_size; + u32 opb_wr_top_line_chroma_buffer_size; + u32 opb_wr_top_line_luma_buffer_size; + u32 macrotiling_size = 32, size; + + opb_wr_top_line_luma_buffer_size = + ALIGN(frame_width, macrotiling_size) / + macrotiling_size * 256; + opb_wr_top_line_luma_buffer_size = + ALIGN(opb_wr_top_line_luma_buffer_size, DMA_ALIGNMENT) + + (MAX_TILE_COLUMNS - 1) * 256; + opb_wr_top_line_luma_buffer_size = + max_t(u32, opb_wr_top_line_luma_buffer_size, + (32 * ALIGN(frame_height, 8))); + opb_wr_top_line_chroma_buffer_size = + opb_wr_top_line_luma_buffer_size; + opb_lb_wr_llb_uv_buffer_size = + ALIGN((ALIGN(frame_height, 8) / (4 / 2)) * 64, + BUFFER_ALIGNMENT_32_BYTES); + opb_lb_wr_llb_y_buffer_size = + ALIGN((ALIGN(frame_height, 8) / (4 / 2)) * 64, + BUFFER_ALIGNMENT_32_BYTES); + size = num_vpp_pipes * 2 * + (vpss_4tap_top_buffer_size + vpss_div2_top_buffer_size) + + 2 * (vpss_4tap_left_buffer_size + vpss_div2_left_buffer_size) + + opb_wr_top_line_luma_buffer_size + + opb_wr_top_line_chroma_buffer_size + + opb_lb_wr_llb_uv_buffer_size + + opb_lb_wr_llb_y_buffer_size; + + return size; +} + +static inline +u32 hfi_buffer_line_h264d(u32 frame_width, u32 frame_height, + bool is_opb, u32 num_vpp_pipes) +{ + u32 vpss_lb_size = 0; + u32 _size; + + _size = ALIGN(size_h264d_lb_fe_top_data(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h264d_lb_fe_top_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h264d_lb_fe_left_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) * num_vpp_pipes + + ALIGN(size_h264d_lb_se_top_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h264d_lb_se_left_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) * num_vpp_pipes + + ALIGN(size_h264d_lb_pe_top_data(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h264d_lb_vsp_top(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h264d_lb_recon_dma_metadata_wr(frame_width, frame_height), + DMA_ALIGNMENT) * 2 + + ALIGN(size_h264d_qp(frame_width, frame_height), + DMA_ALIGNMENT); + _size = ALIGN(_size, DMA_ALIGNMENT); + if (is_opb) + vpss_lb_size = size_vpss_lb(frame_width, frame_height, + num_vpp_pipes); + + _size = ALIGN((_size + vpss_lb_size), + DMA_ALIGNMENT); + + return _size; +} + +static inline +u32 size_h265d_lb_fe_top_data(u32 frame_width, u32 frame_height) +{ + return MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE * + (ALIGN(frame_width, 64) + 8) * 2; +} + +static inline +u32 size_h265d_lb_fe_top_ctrl(u32 frame_width, u32 frame_height) +{ + return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * + (ALIGN(frame_width, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS); +} + +static inline +u32 size_h265d_lb_fe_left_ctrl(u32 frame_width, u32 frame_height) +{ + return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * + (ALIGN(frame_height, LCU_MAX_SIZE_PELS) / LCU_MIN_SIZE_PELS); +} + +static inline +u32 size_h265d_lb_se_top_ctrl(u32 frame_width, u32 frame_height) +{ + return (LCU_MAX_SIZE_PELS / 8 * (128 / 8)) * ((frame_width + 15) >> 4); +} + +static inline +u32 size_h265d_lb_se_left_ctrl(u32 frame_width, u32 frame_height) +{ + return max_t(u32, ((frame_height + 16 - 1) / 8) * + MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE, + max_t(u32, ((frame_height + 32 - 1) / 8) * + MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE, + ((frame_height + 64 - 1) / 8) * + MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE)); +} + +static inline +u32 size_h265d_lb_pe_top_data(u32 frame_width, u32 frame_height) +{ + return MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE * + (ALIGN(frame_width, LCU_MIN_SIZE_PELS) / LCU_MIN_SIZE_PELS); +} + +static inline +u32 size_h265d_lb_vsp_top(u32 frame_width, u32 frame_height) +{ + return ((frame_width + 63) >> 6) * 128; +} + +static inline +u32 size_h265d_lb_vsp_left(u32 frame_width, u32 frame_height) +{ + return ((frame_height + 63) >> 6) * 128; +} + +static inline +u32 size_h265d_lb_recon_dma_metadata_wr(u32 frame_width, u32 frame_height) +{ + return size_h264d_lb_recon_dma_metadata_wr(frame_width, frame_height); +} + +static inline +u32 size_h265d_qp(u32 frame_width, u32 frame_height) +{ + return size_h264d_qp(frame_width, frame_height); +} + +static inline +u32 hfi_buffer_line_h265d(u32 frame_width, u32 frame_height, + bool is_opb, u32 num_vpp_pipes) +{ + u32 vpss_lb_size = 0, _size; + + _size = ALIGN(size_h265d_lb_fe_top_data(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h265d_lb_fe_top_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h265d_lb_fe_left_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) * num_vpp_pipes + + ALIGN(size_h265d_lb_se_left_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) * num_vpp_pipes + + ALIGN(size_h265d_lb_se_top_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h265d_lb_pe_top_data(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h265d_lb_vsp_top(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_h265d_lb_vsp_left(frame_width, frame_height), + DMA_ALIGNMENT) * num_vpp_pipes + + ALIGN(size_h265d_lb_recon_dma_metadata_wr(frame_width, frame_height), + DMA_ALIGNMENT) * 4 + + ALIGN(size_h265d_qp(frame_width, frame_height), + DMA_ALIGNMENT); + if (is_opb) + vpss_lb_size = size_vpss_lb(frame_width, frame_height, num_vpp_pipes); + + return ALIGN((_size + vpss_lb_size), DMA_ALIGNMENT); +} + +static inline +u32 size_vpxd_lb_fe_left_ctrl(u32 frame_width, u32 frame_height) +{ + return max_t(u32, ((frame_height + 15) >> 4) * MAX_FE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE, + max_t(u32, ((frame_height + 31) >> 5) * MAX_FE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE, + ((frame_height + 63) >> 6) * MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE)); +} + +static inline +u32 size_vpxd_lb_fe_top_ctrl(u32 frame_width, u32 frame_height) +{ + return ((ALIGN(frame_width, 64) + 8) * 10 * 2); +} + +static inline +u32 size_vpxd_lb_se_top_ctrl(u32 frame_width, u32 frame_height) +{ + return ((frame_width + 15) >> 4) * MAX_FE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE; +} + +static inline +u32 size_vpxd_lb_se_left_ctrl(u32 frame_width, u32 frame_height) +{ + return max_t(u32, ((frame_height + 15) >> 4) * MAX_SE_NBR_CTRL_LCU16_LINE_BUFFER_SIZE, + max_t(u32, ((frame_height + 31) >> 5) * MAX_SE_NBR_CTRL_LCU32_LINE_BUFFER_SIZE, + ((frame_height + 63) >> 6) * MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE)); +} + +static inline +u32 size_vpxd_lb_recon_dma_metadata_wr(u32 frame_width, u32 frame_height) +{ + return ALIGN((ALIGN(frame_height, 8) / (4 / 2)) * 64, + BUFFER_ALIGNMENT_32_BYTES); +} + +static inline +u32 size_mp2d_lb_fe_top_data(u32 frame_width, u32 frame_height) +{ + return ((ALIGN(frame_width, 16) + 8) * 10 * 2); +} + +static inline +u32 size_vp9d_lb_fe_top_data(u32 frame_width, u32 frame_height) +{ + return (ALIGN(ALIGN(frame_width, 8), 64) + 8) * 10 * 2; +} + +static inline +u32 size_vp9d_lb_pe_top_data(u32 frame_width, u32 frame_height) +{ + return ((ALIGN(ALIGN(frame_width, 8), 64) >> 6) * 176); +} + +static inline +u32 size_vp9d_lb_vsp_top(u32 frame_width, u32 frame_height) +{ + return (((ALIGN(ALIGN(frame_width, 8), 64) >> 6) * 64 * 8) + 256); +} + +static inline u32 hfi_iris3_vp9d_comv_size(void) +{ + return (((8192 + 63) >> 6) * ((4320 + 63) >> 6) * 8 * 8 * 2 * 8); +} + +static inline +u32 size_vp9d_qp(u32 frame_width, u32 frame_height) +{ + return size_h264d_qp(frame_width, frame_height); +} + +static inline +u32 hfi_iris3_vp9d_lb_size(u32 frame_width, u32 frame_height, + u32 num_vpp_pipes) +{ + return ALIGN(size_vpxd_lb_fe_left_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) * num_vpp_pipes + + ALIGN(size_vpxd_lb_se_left_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) * num_vpp_pipes + + ALIGN(size_vp9d_lb_vsp_top(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_vpxd_lb_fe_top_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) + + 2 * ALIGN(size_vpxd_lb_recon_dma_metadata_wr(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_vpxd_lb_se_top_ctrl(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_vp9d_lb_pe_top_data(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_vp9d_lb_fe_top_data(frame_width, frame_height), + DMA_ALIGNMENT) + + ALIGN(size_vp9d_qp(frame_width, frame_height), + DMA_ALIGNMENT); +} + +static inline +u32 hfi_buffer_line_vp9d(u32 frame_width, u32 frame_height, + u32 _yuv_bufcount_min, bool is_opb, + u32 num_vpp_pipes) +{ + u32 vpss_lb_size = 0; + u32 _lb_size = 0; + + _lb_size = hfi_iris3_vp9d_lb_size(frame_width, frame_height, + num_vpp_pipes); + + if (is_opb) + vpss_lb_size = size_vpss_lb(frame_width, frame_height, + num_vpp_pipes); + + return _lb_size + vpss_lb_size; +} + +static inline u32 hfi_buffer_persist_h264d(u32 rpu_enabled) +{ + return ALIGN(SIZE_SLIST_BUF_H264 * NUM_SLIST_BUF_H264 + + H264_DISPLAY_BUF_SIZE * H264_NUM_FRM_INFO + + NUM_HW_PIC_BUF * SIZE_SEI_USERDATA + + (rpu_enabled) * NUM_HW_PIC_BUF * SIZE_DOLBY_RPU_METADATA, + DMA_ALIGNMENT); +} + +static inline +u32 hfi_buffer_persist_h265d(u32 rpu_enabled) +{ + return ALIGN((SIZE_SLIST_BUF_H265 * NUM_SLIST_BUF_H265 + + H265_NUM_FRM_INFO * H265_DISPLAY_BUF_SIZE + + H265_NUM_TILE * sizeof(u32) + + NUM_HW_PIC_BUF * SIZE_SEI_USERDATA + + rpu_enabled * NUM_HW_PIC_BUF * SIZE_DOLBY_RPU_METADATA), + DMA_ALIGNMENT); +} + +static inline u32 hfi_buffer_persist_vp9d(void) +{ + return ALIGN(VP9_NUM_PROBABILITY_TABLE_BUF * VP9_PROB_TABLE_SIZE, + DMA_ALIGNMENT) + + ALIGN(hfi_iris3_vp9d_comv_size(), DMA_ALIGNMENT) + + ALIGN(MAX_SUPERFRAME_HEADER_LEN, DMA_ALIGNMENT) + + ALIGN(VP9_UDC_HEADER_BUF_SIZE, DMA_ALIGNMENT) + + ALIGN(VP9_NUM_FRAME_INFO_BUF * CCE_TILE_OFFSET_SIZE, + DMA_ALIGNMENT) + + ALIGN(VP9_NUM_FRAME_INFO_BUF * VP9_FRAME_INFO_BUF_SIZE, + DMA_ALIGNMENT) + + HDR10_HIST_EXTRADATA_SIZE; +} + +static inline +u32 hfi_nv12_ubwc_il_calc_y_buf_size(u32 frame_width, u32 frame_height, + u32 stride_multiple, + u32 min_buf_height_multiple) +{ + u32 stride, buf_height; + + stride = ALIGN(frame_width, stride_multiple); + buf_height = ALIGN(frame_height, min_buf_height_multiple); + + return ALIGN(stride * buf_height, HFI_ALIGNMENT_4096); +} + +static inline +u32 hfi_nv12_ubwc_il_calc_uv_buf_size(u32 frame_width, u32 frame_height, + u32 stride_multiple, + u32 min_buf_height_multiple) +{ + u32 uv_stride, uv_buf_height; + + uv_stride = ALIGN(frame_width, stride_multiple); + uv_buf_height = ALIGN(((frame_height + 1) >> 1), + min_buf_height_multiple); + + return ALIGN(uv_stride * uv_buf_height, HFI_ALIGNMENT_4096); +} + +static inline +u32 hfi_ubwc_calc_metadata_plane_stride(u32 frame_width, + u32 metadata_stride_multiple, + u32 tile_width_in_pels) +{ + return ALIGN(((frame_width + (tile_width_in_pels - 1)) / tile_width_in_pels), + metadata_stride_multiple); +} + +static inline +u32 hfi_ubwc_metadata_plane_bufheight(u32 frame_height, + u32 metadata_height_multiple, + u32 tile_height_in_pels) +{ + return ALIGN(((frame_height + (tile_height_in_pels - 1)) / tile_height_in_pels), + metadata_height_multiple); +} + +static inline +u32 hfi_ubwc_uv_metadata_plane_stride(u32 frame_width, + u32 metadata_stride_multiple, + u32 tile_width_in_pels) +{ + return ALIGN(((((frame_width + 1) >> 1) + tile_width_in_pels - 1) / + tile_width_in_pels), metadata_stride_multiple); +} + +static inline +u32 hfi_ubwc_uv_metadata_plane_bufheight(u32 frame_height, + u32 metadata_height_multiple, + u32 tile_height_in_pels) +{ + return ALIGN(((((frame_height + 1) >> 1) + tile_height_in_pels - 1) / + tile_height_in_pels), metadata_height_multiple); +} + +static inline +u32 hfi_ubwc_metadata_plane_buffer_size(u32 _metadata_tride, u32 _metadata_buf_height) +{ + return ALIGN(_metadata_tride * _metadata_buf_height, HFI_ALIGNMENT_4096); +} + +static inline +u32 hfi_nv12_ubwc_il_calc_buf_size_v2(u32 frame_width, + u32 frame_height, + u32 y_stride_multiple, + u32 y_buffer_height_multiple, + u32 uv_stride_multiple, + u32 uv_buffer_height_multiple, + u32 y_metadata_stride_multiple, + u32 y_metadata_buffer_height_multiple, + u32 uv_metadata_stride_multiple, + u32 uv_metadata_buffer_height_multiple) +{ + u32 y_buf_size, uv_buf_size, y_meta_size, uv_meta_size; + u32 half_height = (frame_height + 1) >> 1; + u32 stride, _height; + + y_buf_size = + hfi_nv12_ubwc_il_calc_y_buf_size(frame_width, half_height, + y_stride_multiple, + y_buffer_height_multiple); + uv_buf_size = + hfi_nv12_ubwc_il_calc_uv_buf_size(frame_width, half_height, + uv_stride_multiple, + uv_buffer_height_multiple); + stride = + hfi_ubwc_calc_metadata_plane_stride(frame_width, + y_metadata_stride_multiple, + HFI_COL_FMT_NV12C_Y_TILE_WIDTH); + _height = + hfi_ubwc_metadata_plane_bufheight(half_height, + y_metadata_buffer_height_multiple, + HFI_COL_FMT_NV12C_Y_TILE_HEIGHT); + y_meta_size = hfi_ubwc_metadata_plane_buffer_size(stride, _height); + stride = + hfi_ubwc_uv_metadata_plane_stride(frame_width, + uv_metadata_stride_multiple, + HFI_COL_FMT_NV12C_UV_TILE_WIDTH); + _height = + hfi_ubwc_uv_metadata_plane_bufheight(half_height, + uv_metadata_buffer_height_multiple, + HFI_COL_FMT_NV12C_UV_TILE_HEIGHT); + uv_meta_size = hfi_ubwc_metadata_plane_buffer_size(stride, _height); + + return (y_buf_size + uv_buf_size + y_meta_size + uv_meta_size) << 1; +} + +static inline +u32 hfi_yuv420_tp10_ubwc_calc_y_buf_size(u32 y_stride, + u32 y_buf_height) +{ + return ALIGN(y_stride * y_buf_height, HFI_ALIGNMENT_4096); +} + +static inline +u32 hfi_yuv420_tp10_ubwc_calc_uv_buf_size(u32 uv_stride, + u32 uv_buf_height) +{ + return ALIGN(uv_stride * uv_buf_height, HFI_ALIGNMENT_4096); +} + +static inline +u32 hfi_yuv420_tp10_ubwc_calc_buf_size(u32 y_stride, u32 y_buf_height, + u32 uv_stride, u32 uv_buf_height, + u32 y_md_stride, u32 y_md_height, + u32 uv_md_stride, u32 uv_md_height) +{ + u32 y_data_size, uv_data_size, y_md_size, uv_md_size; + + y_data_size = hfi_yuv420_tp10_ubwc_calc_y_buf_size(y_stride, y_buf_height); + uv_data_size = hfi_yuv420_tp10_ubwc_calc_uv_buf_size(uv_stride, uv_buf_height); + y_md_size = hfi_ubwc_metadata_plane_buffer_size(y_md_stride, y_md_height); + uv_md_size = hfi_ubwc_metadata_plane_buffer_size(uv_md_stride, uv_md_height); + + return y_data_size + uv_data_size + y_md_size + uv_md_size; +} + +int iris_int_buf_size_iris3(struct iris_inst *inst, + enum iris_buffer_type buffer_type); + +#endif From patchwork Mon Dec 18 11:32:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755792 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C21AA1C698; Mon, 18 Dec 2023 11:40:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="TGLaTVKb" Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIArvvC003521; Mon, 18 Dec 2023 11:40:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=qhSUy+6TOTvWGOHRrcWbcV5KaOQ6iVEmTghe6XRp+Lg=; b=TG LaTVKb/KVPbaFMXKQ6a+OKqzri/hylahTK5DtWB5wMNVO60nN9C4IwxKM3eA3Zun 9ki6bCuCcvsd+kuhbjN3aOe8d6EngcJQDnnR0QHecys6h6uWF0QHyHupPUj28M4a b3aEXt+kd3GLpBO5XCWMWCAOHu2Fe7Adb3BReTfQYIvmTrUgKTcAsI/eNRFSDupG oCAgeuk2thV8REtKHBKzPIibAeTTLqNrwmtxC47Wuj79UDC23RVseYUyX23HpYqr QgliLZcYSk6vuW1AVjztrTodgoTcjntwibV8dmlp6ziG3wKPuN+XBoHom5cat916 H7nUlExdye6CW+43sdfw== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mb9g4xy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:03 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBe0ww004854; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ag-2; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7pR030023; Mon, 18 Dec 2023 11:33:08 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6B7029967; Mon, 18 Dec 2023 11:33:08 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 55BF1232E; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 22/34] media: iris: introduce instance states Date: Mon, 18 Dec 2023 17:02:17 +0530 Message-Id: <1702899149-21321-23-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: X7JjZwzpVHK3NnlkLrxR0-h3_3ESkyBr X-Proofpoint-GUID: X7JjZwzpVHK3NnlkLrxR0-h3_3ESkyBr X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 impostorscore=0 adultscore=0 mlxscore=0 clxscore=1015 spamscore=0 mlxlogscore=944 phishscore=0 lowpriorityscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Introduce different states for instances. State transitions are defined, based on which they would be allowed/rejected/ignored. IRIS_INST_OPEN - video instance is opened. IRIS_INST_INPUT_STREAMING - stream on is completed on output plane. IRIS_INST_OUTPUT_STREAMING - stream on is completed on capture plane. IRIS_INST_STREAMING - stream on is completed on both output and capture planes. IRIS_INST_CLOSE - video instance is closed. IRIS_INST_ERROR - error state. | v ------------- +---------| OPEN |--------- + | ------------- | | ^ ^ | | / \ | | / \ | | v v | | ----------- ----------- | | | INPUT OUTPUT | | |---| STREAMING STREAMING |---| | ----------- ----------- | | ^ ^ | | \ / | | \ / | | v v | | ------------- | |--------| STREAMING |-----------| | ------------- | | | | | | | | v | | ----------- | +-------->| CLOSE |<----------+ | ----------- | | | | | | | | v | | ---------- | +-------->| ERROR |<-----------+ ---------- Signed-off-by: Dikshita Agarwal --- .../platform/qcom/vcodec/iris/iris_instance.h | 2 + .../media/platform/qcom/vcodec/iris/iris_state.c | 86 ++++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_state.h | 22 ++++++ drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 3 + .../media/platform/qcom/vcodec/iris/iris_vidc.c | 5 ++ 5 files changed, 118 insertions(+) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h index 5d4c856..275efa8 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -37,6 +37,7 @@ * @codec: codec type * @buffers: structure of buffer info * @fw_min_count: minimnum count of buffers needed by fw + * @state: instance state */ struct iris_inst { @@ -60,6 +61,7 @@ struct iris_inst { enum codec_type codec; struct iris_buffers_info buffers; u32 fw_min_count; + enum iris_inst_state state; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c index 83bbc6b..9bf79a0 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c @@ -69,3 +69,89 @@ int iris_change_core_state(struct iris_core *core, return ret; } + +struct iris_inst_state_change_allow { + enum iris_inst_state from; + enum iris_inst_state to; + enum state_change allow; +}; + +static enum state_change iris_allow_inst_state_change(struct iris_inst *inst, + enum iris_inst_state req_state) +{ + enum state_change allow = STATE_CHANGE_ALLOW; + int cnt; + static struct iris_inst_state_change_allow state[] = { + /* from, to, allow */ + {IRIS_INST_OPEN, IRIS_INST_OPEN, STATE_CHANGE_IGNORE }, + {IRIS_INST_OPEN, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_ALLOW }, + {IRIS_INST_OPEN, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_ALLOW }, + {IRIS_INST_OPEN, IRIS_INST_STREAMING, STATE_CHANGE_DISALLOW }, + {IRIS_INST_OPEN, IRIS_INST_CLOSE, STATE_CHANGE_ALLOW }, + {IRIS_INST_OPEN, IRIS_INST_ERROR, STATE_CHANGE_ALLOW }, + + {IRIS_INST_INPUT_STREAMING, IRIS_INST_OPEN, STATE_CHANGE_ALLOW }, + {IRIS_INST_INPUT_STREAMING, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_IGNORE }, + {IRIS_INST_INPUT_STREAMING, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_DISALLOW }, + {IRIS_INST_INPUT_STREAMING, IRIS_INST_STREAMING, STATE_CHANGE_ALLOW }, + {IRIS_INST_INPUT_STREAMING, IRIS_INST_CLOSE, STATE_CHANGE_ALLOW }, + {IRIS_INST_INPUT_STREAMING, IRIS_INST_ERROR, STATE_CHANGE_ALLOW }, + + {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_OPEN, STATE_CHANGE_ALLOW }, + {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_DISALLOW }, + {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_IGNORE }, + {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_STREAMING, STATE_CHANGE_ALLOW }, + {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_CLOSE, STATE_CHANGE_ALLOW }, + {IRIS_INST_OUTPUT_STREAMING, IRIS_INST_ERROR, STATE_CHANGE_ALLOW }, + + {IRIS_INST_STREAMING, IRIS_INST_OPEN, STATE_CHANGE_DISALLOW }, + {IRIS_INST_STREAMING, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_ALLOW }, + {IRIS_INST_STREAMING, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_ALLOW }, + {IRIS_INST_STREAMING, IRIS_INST_STREAMING, STATE_CHANGE_IGNORE }, + {IRIS_INST_STREAMING, IRIS_INST_CLOSE, STATE_CHANGE_ALLOW }, + {IRIS_INST_STREAMING, IRIS_INST_ERROR, STATE_CHANGE_ALLOW }, + + {IRIS_INST_CLOSE, IRIS_INST_OPEN, STATE_CHANGE_DISALLOW }, + {IRIS_INST_CLOSE, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_DISALLOW }, + {IRIS_INST_CLOSE, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_DISALLOW }, + {IRIS_INST_CLOSE, IRIS_INST_STREAMING, STATE_CHANGE_DISALLOW }, + {IRIS_INST_CLOSE, IRIS_INST_CLOSE, STATE_CHANGE_IGNORE }, + {IRIS_INST_CLOSE, IRIS_INST_ERROR, STATE_CHANGE_IGNORE }, + + {IRIS_INST_ERROR, IRIS_INST_OPEN, STATE_CHANGE_IGNORE }, + {IRIS_INST_ERROR, IRIS_INST_INPUT_STREAMING, STATE_CHANGE_IGNORE }, + {IRIS_INST_ERROR, IRIS_INST_OUTPUT_STREAMING, STATE_CHANGE_IGNORE }, + {IRIS_INST_ERROR, IRIS_INST_STREAMING, STATE_CHANGE_IGNORE }, + {IRIS_INST_ERROR, IRIS_INST_CLOSE, STATE_CHANGE_IGNORE }, + {IRIS_INST_ERROR, IRIS_INST_ERROR, STATE_CHANGE_IGNORE }, + }; + + for (cnt = 0; cnt < ARRAY_SIZE(state); cnt++) { + if (state[cnt].from == inst->state && state[cnt].to == req_state) { + allow = state[cnt].allow; + break; + } + } + + return allow; +} + +int iris_inst_change_state(struct iris_inst *inst, + enum iris_inst_state request_state) +{ + enum state_change allow; + + if (IS_SESSION_ERROR(inst)) + return 0; + + if (inst->state == request_state) + return 0; + + allow = iris_allow_inst_state_change(inst, request_state); + if (allow != STATE_CHANGE_ALLOW) + return (allow == STATE_CHANGE_DISALLOW ? -EINVAL : 0); + + inst->state = request_state; + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h index ee20842..6db95a1 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h @@ -7,6 +7,7 @@ #define _IRIS_STATE_H_ struct iris_core; +struct iris_inst; enum iris_core_state { IRIS_CORE_DEINIT, @@ -15,8 +16,29 @@ enum iris_core_state { IRIS_CORE_ERROR, }; +enum iris_inst_state { + IRIS_INST_OPEN, + IRIS_INST_INPUT_STREAMING, + IRIS_INST_OUTPUT_STREAMING, + IRIS_INST_STREAMING, + IRIS_INST_CLOSE, + IRIS_INST_ERROR, +}; + +enum state_change { + STATE_CHANGE_ALLOW, + STATE_CHANGE_DISALLOW, + STATE_CHANGE_IGNORE, +}; + +#define IS_SESSION_ERROR(inst) \ +((inst)->state == IRIS_INST_ERROR) + bool core_in_valid_state(struct iris_core *core); int iris_change_core_state(struct iris_core *core, enum iris_core_state request_state); +int iris_inst_change_state(struct iris_inst *inst, + enum iris_inst_state request_state); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c index a57b5fb..8f499a9 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c @@ -34,6 +34,9 @@ int iris_vb2_queue_setup(struct vb2_queue *q, else f = inst->fmt_dst; + if (inst->state == IRIS_INST_STREAMING) + return -EINVAL; + if (*num_planes) { if (*num_planes != f->fmt.pix_mp.num_planes || sizes[0] < f->fmt.pix_mp.plane_fmt[0].sizeimage) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c index 124333a..68ba75f 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -168,6 +168,7 @@ int vidc_open(struct file *filp) inst->core = core; inst->session_id = hash32_ptr(inst); + iris_inst_change_state(inst, IRIS_INST_OPEN); mutex_init(&inst->ctx_q_lock); ret = vidc_add_session(inst); @@ -244,6 +245,7 @@ int vidc_close(struct file *filp) v4l2_ctrl_handler_free(&inst->ctrl_handler); vdec_inst_deinit(inst); close_session(inst); + iris_inst_change_state(inst, IRIS_INST_CLOSE); vidc_vb2_queue_deinit(inst); vidc_v4l2_fh_deinit(inst); vidc_remove_session(inst); @@ -295,6 +297,9 @@ static __poll_t vidc_poll(struct file *filp, struct poll_table_struct *pt) if (!inst) return EPOLLERR; + if (IS_SESSION_ERROR(inst)) + return EPOLLERR; + poll_wait(filp, &inst->fh.wait, pt); poll_wait(filp, &inst->vb2q_src->done_wq, pt); poll_wait(filp, &inst->vb2q_dst->done_wq, pt); From patchwork Mon Dec 18 11:32:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755793 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2DC261B272; Mon, 18 Dec 2023 11:36:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="o7WcZUgd" Received: from pps.filterd (m0279865.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIBDcoC018040; Mon, 18 Dec 2023 11:36:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=Q03KspE6e+LrgwjDhfZcLcqLHb5nu3RQ3Bq6dmATH3A=; b=o7 WcZUgdsmECrQYua8jt+o82o2lOwgf5OxfMCF/Nt+3MlAxc0e3TbELuHnjLuFsOba 9d4DJnICVVvwRD8M/E+xDW0W206PQ+GQYZMzQrqbfKp0/CaW9XC93mhLTN3EeoDB qntx/IktP86E5kZj6+9W6A3Y3JyB+JjLJbmeplx5JUeC7GTIM1PuXsN8WMqSRDG/ irb0Th6zn/6zqSCIGLU8R1QEvoqflRBbAN3vMrcmSsU9ET7wE7Izw9BdK8oQb+WD hByaLabj7+SmlHF0enS8S76v+pPTk9xUq8fU2kH37cpLKitxVmD8Yt34p419mXh/ aJwLSMUaHE2YN5wHD1MQ== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2n1781gj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:36:13 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX5R2029941; Mon, 18 Dec 2023 11:36:09 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ykyyva-1; Mon, 18 Dec 2023 11:36:09 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX7pT030023; Mon, 18 Dec 2023 11:36:09 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX8SW030061; Mon, 18 Dec 2023 11:36:09 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 5A85B2346; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 24/34] media: iris: subscribe input and output properties to firmware Date: Mon, 18 Dec 2023 17:02:19 +0530 Message-Id: <1702899149-21321-25-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: YOXUHpHcWpeRvUFeQefdpbGqd7L_sRey X-Proofpoint-GUID: YOXUHpHcWpeRvUFeQefdpbGqd7L_sRey X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 mlxscore=0 lowpriorityscore=0 impostorscore=0 bulkscore=0 adultscore=0 clxscore=1015 priorityscore=1501 suspectscore=0 malwarescore=0 phishscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180083 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Vikash Garodia Driver can subscribe to different properties for which it expects a response from firmware. Different SOCs firmware can support different properties which can be subscribed by driver. HFI_CMD_SUBSCRIBE_MODE with HFI_MODE_PROPERTY - to subscribe any property to firmware. Also, set below mandatory properties on capture plane: HFI_PROP_COLOR_FORMAT HFI_PROP_LINEAR_STRIDE_SCANLINE HFI_PROP_UBWC_STRIDE_SCANLINE. Signed-off-by: Vikash Garodia Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/hfi_defines.h | 31 +++++ drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 38 +++++ drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 3 + .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 42 ++++++ .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 3 + .../media/platform/qcom/vcodec/iris/iris_vdec.c | 154 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_vdec.h | 2 + .../platform/qcom/vcodec/iris/platform_common.h | 8 ++ .../platform/qcom/vcodec/iris/platform_sm8550.c | 29 ++++ 9 files changed, 310 insertions(+) diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index ce4eaff..a6078a5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -25,6 +25,29 @@ #define HFI_PROP_UBWC_BANK_SWZL_LEVEL3 0x03000008 #define HFI_PROP_UBWC_BANK_SPREADING 0x03000009 +enum hfi_property_mode_type { + HFI_MODE_NONE = 0x00000000, + HFI_MODE_PORT_SETTINGS_CHANGE = 0x00000001, + HFI_MODE_PROPERTY = 0x00000002, +}; + +#define HFI_CMD_SUBSCRIBE_MODE 0x0100000B + +enum hfi_color_format { + HFI_COLOR_FMT_OPAQUE = 0, + HFI_COLOR_FMT_NV12 = 1, + HFI_COLOR_FMT_NV12_UBWC = 2, + HFI_COLOR_FMT_P010 = 3, + HFI_COLOR_FMT_TP10_UBWC = 4, + HFI_COLOR_FMT_RGBA8888 = 5, + HFI_COLOR_FMT_RGBA8888_UBWC = 6, + HFI_COLOR_FMT_NV21 = 7, +}; + +#define HFI_PROP_COLOR_FORMAT 0x03000101 + +#define HFI_PROP_LINEAR_STRIDE_SCANLINE 0x03000104 + #define HFI_PROP_PROFILE 0x03000107 #define HFI_PROP_LEVEL 0x03000108 @@ -49,10 +72,18 @@ #define HFI_PROP_DECODE_ORDER_OUTPUT 0x0300015b +#define HFI_PROP_PICTURE_TYPE 0x03000162 + #define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168 #define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169 +#define HFI_PROP_NO_OUTPUT 0x0300016a + +#define HFI_PROP_DPB_LIST 0x0300017A + +#define HFI_PROP_UBWC_STRIDE_SCANLINE 0x03000190 + #define HFI_PROP_COMV_BUFFER_COUNT 0x03000193 #define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001 diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index 8f1e456..d15ce5a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -219,6 +219,44 @@ int iris_hfi_session_open(struct iris_inst *inst) return ret; } +int iris_hfi_session_subscribe_mode(struct iris_inst *inst, + u32 cmd, u32 plane, u32 payload_type, + void *payload, u32 payload_size) +{ + struct iris_core *core; + int ret; + + if (!inst->packet) + return -EINVAL; + + core = inst->core; + mutex_lock(&core->lock); + + if (!validate_session(core, inst)) { + ret = -EINVAL; + goto unlock; + } + + ret = hfi_packet_session_command(inst, + cmd, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED), + get_hfi_port(plane), + inst->session_id, + payload_type, + payload, + payload_size); + if (ret) + goto unlock; + + ret = iris_hfi_queue_cmd_write(inst->core, inst->packet); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + int iris_hfi_session_close(struct iris_inst *inst) { struct iris_core *core; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h index ca2339e..d6b3fca 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h @@ -13,6 +13,9 @@ int iris_hfi_core_init(struct iris_core *core); int iris_hfi_core_deinit(struct iris_core *core); int iris_hfi_session_open(struct iris_inst *inst); int iris_hfi_session_close(struct iris_inst *inst); +int iris_hfi_session_subscribe_mode(struct iris_inst *inst, + u32 cmd, u32 plane, u32 payload_type, + void *payload, u32 payload_size); int iris_hfi_set_property(struct iris_inst *inst, u32 packet_type, u32 flag, u32 plane, u32 payload_type, void *payload, u32 payload_size); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c index a3544d8..d4cdfcf 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -34,6 +34,24 @@ u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type) return hfi_port; } +u32 get_hfi_port(u32 plane) +{ + u32 hfi_port = HFI_PORT_NONE; + + switch (plane) { + case INPUT_MPLANE: + hfi_port = HFI_PORT_BITSTREAM; + break; + case OUTPUT_MPLANE: + hfi_port = HFI_PORT_RAW; + break; + default: + break; + } + + return hfi_port; +} + static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type) { switch (buffer_type) { @@ -58,6 +76,30 @@ static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type) } } +u32 get_hfi_colorformat(u32 colorformat) +{ + u32 hfi_colorformat = HFI_COLOR_FMT_NV12_UBWC; + + switch (colorformat) { + case V4L2_PIX_FMT_NV12: + hfi_colorformat = HFI_COLOR_FMT_NV12; + break; + case V4L2_PIX_FMT_QC08C: + hfi_colorformat = HFI_COLOR_FMT_NV12_UBWC; + break; + case V4L2_PIX_FMT_QC10C: + hfi_colorformat = HFI_COLOR_FMT_TP10_UBWC; + break; + case V4L2_PIX_FMT_NV21: + hfi_colorformat = HFI_COLOR_FMT_NV21; + break; + default: + break; + } + + return hfi_colorformat; +} + int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf) { memset(buf, 0, sizeof(*buf)); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h index bf18553..4276d6d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -81,6 +81,9 @@ enum hfi_packet_port_type { u32 get_hfi_port_from_buffer_type(enum iris_buffer_type buffer_type); int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf); +u32 get_hfi_colorformat(u32 colorformat); +u32 get_hfi_port(u32 plane); + int hfi_packet_sys_init(struct iris_core *core, u8 *pkt, u32 pkt_size); int hfi_packet_image_version(struct iris_core *core, diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index 64067d5..7d16c96 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -5,10 +5,13 @@ #include +#include "hfi_defines.h" #include "iris_buffer.h" #include "iris_common.h" #include "iris_ctrls.h" #include "iris_helpers.h" +#include "iris_hfi.h" +#include "iris_hfi_packet.h" #include "iris_vdec.h" static int vdec_codec_change(struct iris_inst *inst, u32 v4l2_codec) @@ -329,3 +332,154 @@ int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscri return ret; } + +int vdec_subscribe_property(struct iris_inst *inst, u32 plane) +{ + const u32 *subcribe_prop = NULL; + u32 subscribe_prop_size = 0; + struct iris_core *core; + u32 payload[32] = {0}; + u32 i; + + core = inst->core; + + payload[0] = HFI_MODE_PROPERTY; + + if (plane == INPUT_MPLANE) { + subscribe_prop_size = core->platform_data->dec_input_prop_size; + subcribe_prop = core->platform_data->dec_input_prop; + } else if (plane == OUTPUT_MPLANE) { + if (inst->codec == H264) { + subscribe_prop_size = core->platform_data->dec_output_prop_size_avc; + subcribe_prop = core->platform_data->dec_output_prop_avc; + } else if (inst->codec == HEVC) { + subscribe_prop_size = core->platform_data->dec_output_prop_size_hevc; + subcribe_prop = core->platform_data->dec_output_prop_hevc; + } else if (inst->codec == VP9) { + subscribe_prop_size = core->platform_data->dec_output_prop_size_vp9; + subcribe_prop = core->platform_data->dec_output_prop_vp9; + } else { + return -EINVAL; + } + } else { + return -EINVAL; + } + + for (i = 0; i < subscribe_prop_size; i++) + payload[i + 1] = subcribe_prop[i]; + + return iris_hfi_session_subscribe_mode(inst, + HFI_CMD_SUBSCRIBE_MODE, + plane, + HFI_PAYLOAD_U32_ARRAY, + &payload[0], + (subscribe_prop_size + 1) * sizeof(u32)); +} + +static int vdec_set_colorformat(struct iris_inst *inst) +{ + u32 hfi_colorformat; + u32 pixelformat; + + pixelformat = inst->fmt_dst->fmt.pix_mp.pixelformat; + hfi_colorformat = get_hfi_colorformat(pixelformat); + + return iris_hfi_set_property(inst, + HFI_PROP_COLOR_FORMAT, + HFI_HOST_FLAGS_NONE, + get_hfi_port(OUTPUT_MPLANE), + HFI_PAYLOAD_U32, + &hfi_colorformat, + sizeof(u32)); +} + +static int vdec_set_linear_stride_scanline(struct iris_inst *inst) +{ + u32 stride_y, scanline_y, stride_uv, scanline_uv; + u32 pixelformat; + u32 payload[2]; + + pixelformat = inst->fmt_dst->fmt.pix_mp.pixelformat; + + if (!is_linear_colorformat(pixelformat)) + return 0; + + stride_y = inst->fmt_dst->fmt.pix_mp.width; + scanline_y = inst->fmt_dst->fmt.pix_mp.height; + stride_uv = stride_y; + scanline_uv = scanline_y / 2; + + payload[0] = stride_y << 16 | scanline_y; + payload[1] = stride_uv << 16 | scanline_uv; + + return iris_hfi_set_property(inst, + HFI_PROP_LINEAR_STRIDE_SCANLINE, + HFI_HOST_FLAGS_NONE, + get_hfi_port(OUTPUT_MPLANE), + HFI_PAYLOAD_U64, + &payload, + sizeof(u64)); +} + +static int vdec_set_ubwc_stride_scanline(struct iris_inst *inst) +{ + u32 meta_stride_y, meta_scanline_y, meta_stride_uv, meta_scanline_uv; + u32 stride_y, scanline_y, stride_uv, scanline_uv; + u32 pix_fmt, width, height; + u32 payload[4]; + + pix_fmt = inst->fmt_dst->fmt.pix_mp.pixelformat; + width = inst->fmt_dst->fmt.pix_mp.width; + height = inst->fmt_dst->fmt.pix_mp.height; + + if (is_linear_colorformat(pix_fmt)) + return 0; + + if (pix_fmt == V4L2_PIX_FMT_QC08C) { + stride_y = ALIGN(width, 128); + scanline_y = ALIGN(height, 32); + stride_uv = ALIGN(width, 128); + scanline_uv = ALIGN((height + 1) >> 1, 32); + meta_stride_y = ALIGN(DIV_ROUND_UP(width, 32), 64); + meta_scanline_y = ALIGN(DIV_ROUND_UP(height, 8), 16); + meta_stride_uv = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 16), 64); + meta_scanline_uv = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 8), 16); + } else { + stride_y = ALIGN(ALIGN(width, 192) * 4 / 3, 256); + scanline_y = ALIGN(height, 16); + stride_uv = ALIGN(ALIGN(width, 192) * 4 / 3, 256); + scanline_uv = ALIGN((height + 1) >> 1, 16); + meta_stride_y = ALIGN(DIV_ROUND_UP(width, 48), 64); + meta_scanline_y = ALIGN(DIV_ROUND_UP(height, 4), 16); + meta_stride_uv = ALIGN(DIV_ROUND_UP((width + 1) >> 1, 24), 64); + meta_scanline_uv = ALIGN(DIV_ROUND_UP((height + 1) >> 1, 4), 16); + } + + payload[0] = stride_y << 16 | scanline_y; + payload[1] = stride_uv << 16 | scanline_uv; + payload[2] = meta_stride_y << 16 | meta_scanline_y; + payload[3] = meta_stride_uv << 16 | meta_scanline_uv; + + return iris_hfi_set_property(inst, + HFI_PROP_UBWC_STRIDE_SCANLINE, + HFI_HOST_FLAGS_NONE, + get_hfi_port(OUTPUT_MPLANE), + HFI_PAYLOAD_U32_ARRAY, + &payload[0], + sizeof(u32) * 4); +} + +int vdec_set_output_property(struct iris_inst *inst) +{ + int ret; + + ret = vdec_set_colorformat(inst); + if (ret) + return ret; + + ret = vdec_set_linear_stride_scanline(inst); + if (ret) + return ret; + + return vdec_set_ubwc_stride_scanline(inst); +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h index a2f159d..6b0306c 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h @@ -14,5 +14,7 @@ int vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f); int vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f); int vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f); int vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub); +int vdec_subscribe_property(struct iris_inst *inst, u32 plane); +int vdec_set_output_property(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h index abd11fa..fc12bde 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h @@ -206,6 +206,14 @@ struct platform_data { u32 core_data_size; struct plat_inst_cap *inst_cap_data; u32 inst_cap_data_size; + const u32 *dec_input_prop; + unsigned int dec_input_prop_size; + const u32 *dec_output_prop_avc; + unsigned int dec_output_prop_size_avc; + const u32 *dec_output_prop_hevc; + unsigned int dec_output_prop_size_hevc; + const u32 *dec_output_prop_vp9; + unsigned int dec_output_prop_size_vp9; }; int init_platform(struct iris_core *core); diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c index 85bc677..6a4bfa3 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c +++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c @@ -382,6 +382,26 @@ static struct format_capability format_data_sm8550 = { .color_format_info_size = ARRAY_SIZE(color_format_data_sm8550), }; +static const u32 sm8550_vdec_input_properties[] = { + HFI_PROP_NO_OUTPUT, +}; + +static const u32 sm8550_vdec_output_properties_avc[] = { + HFI_PROP_PICTURE_TYPE, + HFI_PROP_DPB_LIST, + HFI_PROP_CABAC_SESSION, +}; + +static const u32 sm8550_vdec_output_properties_hevc[] = { + HFI_PROP_PICTURE_TYPE, + HFI_PROP_DPB_LIST, +}; + +static const u32 sm8550_vdec_output_properties_vp9[] = { + HFI_PROP_PICTURE_TYPE, + HFI_PROP_DPB_LIST, +}; + struct platform_data sm8550_data = { .bus_tbl = sm8550_bus_table, .bus_tbl_size = ARRAY_SIZE(sm8550_bus_table), @@ -409,4 +429,13 @@ struct platform_data sm8550_data = { .inst_cap_data_size = ARRAY_SIZE(instance_cap_data_sm8550), .ubwc_config = ubwc_config_sm8550, .format_data = &format_data_sm8550, + + .dec_input_prop = sm8550_vdec_input_properties, + .dec_input_prop_size = ARRAY_SIZE(sm8550_vdec_input_properties), + .dec_output_prop_avc = sm8550_vdec_output_properties_avc, + .dec_output_prop_size_avc = ARRAY_SIZE(sm8550_vdec_output_properties_avc), + .dec_output_prop_hevc = sm8550_vdec_output_properties_hevc, + .dec_output_prop_size_hevc = ARRAY_SIZE(sm8550_vdec_output_properties_hevc), + .dec_output_prop_vp9 = sm8550_vdec_output_properties_vp9, + .dec_output_prop_size_vp9 = ARRAY_SIZE(sm8550_vdec_output_properties_vp9), }; From patchwork Mon Dec 18 11:32:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755788 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BFD8F200DD; Mon, 18 Dec 2023 11:40:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="NaJLUqwO" Received: from pps.filterd (m0279865.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIBF2DJ020287; Mon, 18 Dec 2023 11:40:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=2B2H49iDz0W9d1l1UKpjzmSYJV0KYD6exsP+wHxqA5A=; b=Na JLUqwOqOT38WkcxXbRkTzJ3c4eb1DabZ/kkfrfqrYfNBXoEZfyrKY9F40Uj5fbGh iZQE6WRrVvPp26WL9eTPIbMHL+7yNP10FkZ9u1sBlUqiMwfciTgQaZg8TJIGfqEW ASUXXqlo1bhLE/OlvHnhStDJB8p1GhRaIL0Q+qR3sY1+/U+LVzFzDlt3V5W4bhHR OfIewkNo0OjEHkIYuChIsUJTXfwNLIOCuzRR51an3shAZ1AzkF9m+1mhDLN4mNEO vFAy+3erTwNN7FD2oRnGyFqoHUDx+5N/eGGqxClrvtmAGo8sOPM9t3+I2W5Q9Q1p EXRxkP0KNzLzoZIZVu0g== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2n1781tv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:04 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBe0Hn004853; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ae-3; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX77m029983; Mon, 18 Dec 2023 11:33:08 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX6J0029963; Mon, 18 Dec 2023 11:33:08 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 620202358; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 27/34] media: iris: implement vb2 ops for buf_queue and firmware response Date: Mon, 18 Dec 2023 17:02:22 +0530 Message-Id: <1702899149-21321-28-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: gK5mxoCh8tKNMrmSfHTLRqXEaSkNTTdK X-Proofpoint-GUID: gK5mxoCh8tKNMrmSfHTLRqXEaSkNTTdK X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 mlxscore=0 lowpriorityscore=0 impostorscore=0 bulkscore=0 adultscore=0 clxscore=1015 priorityscore=1501 suspectscore=0 malwarescore=0 phishscore=0 spamscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Implement vb2 ops for buf queue. Below are the different buffer attributes: BUF_ATTR_DEFERRED - buffer queued by client but not submitted to firmware. BUF_ATTR_READ_ONLY - processed buffer received from firmware as read only. These buffers are held in firmware as reference for future frame processing. BUF_ATTR_PENDING_RELEASE - buffers requested to be released from firmware. BUF_ATTR_QUEUED - buffers submitted to firmware. BUF_ATTR_DEQUEUED - buffers received from firmware. BUF_ATTR_BUFFER_DONE - buffers sent back to vb2. Buffers are submitted and received via HFI_CMD_BUFFER. Firmware associates below flags during buffer response: HFI_BUF_FW_FLAG_RELEASE_DONE - buffer released in firmware. HFI_BUF_FW_FLAG_READONLY - buffer used as reference in firmware. Input buffers dequeued from firmware are sent directly to vb2. Output buffers if read only, are sent to vb2 and also maintained in read only list. If the same read only buffer is received form client, HFI_BUF_HOST_FLAG_READONLY is attached to the buffer and submitted to firmware. Once the buffer is received from firmware as non read only, it is removed from read only list. Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/hfi_defines.h | 6 + .../media/platform/qcom/vcodec/iris/iris_buffer.h | 1 - .../media/platform/qcom/vcodec/iris/iris_common.h | 9 + .../media/platform/qcom/vcodec/iris/iris_helpers.c | 369 +++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_helpers.h | 11 +- .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 28 ++ .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 2 + .../platform/qcom/vcodec/iris/iris_hfi_response.c | 392 +++++++++++++++++++++ .../media/platform/qcom/vcodec/iris/iris_state.c | 8 + .../media/platform/qcom/vcodec/iris/iris_state.h | 1 + drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 81 +++++ drivers/media/platform/qcom/vcodec/iris/iris_vb2.h | 1 + .../media/platform/qcom/vcodec/iris/iris_vdec.c | 49 +++ .../media/platform/qcom/vcodec/iris/iris_vdec.h | 1 + .../media/platform/qcom/vcodec/iris/iris_vidc.c | 2 + 15 files changed, 959 insertions(+), 2 deletions(-) diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index dfd1a4c..104ef9c 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -238,4 +238,10 @@ enum hfi_buffer_host_flags { HFI_BUF_HOST_FLAGS_CB_NON_SECURE = 0x00000200, }; +enum hfi_buffer_firmware_flags { + HFI_BUF_FW_FLAG_NONE = 0x00000000, + HFI_BUF_FW_FLAG_RELEASE_DONE = 0x00000001, + HFI_BUF_FW_FLAG_READONLY = 0x00000010, +}; + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h index 8769c3d..59c4a10 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_buffer.h @@ -58,7 +58,6 @@ int iris_destroy_internal_buffer(struct iris_inst *inst, int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane); int iris_release_input_internal_buffers(struct iris_inst *inst); - int iris_alloc_and_queue_session_int_bufs(struct iris_inst *inst, enum iris_buffer_type buffer_type); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h index 0fbd139..5fd96d3 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -23,6 +23,8 @@ #define NUM_MBS_4k (((4096 + 15) >> 4) * ((2304 + 15) >> 4)) +#define MAX_DPB_COUNT 32 + #define MAX_DPB_LIST_ARRAY_SIZE (16 * 4) #define MAX_DPB_LIST_PAYLOAD_SIZE (16 * 4 * 4) @@ -69,6 +71,13 @@ enum iris_buffer_type { BUF_VPSS, }; +enum iris_buffer_flags { + BUF_FLAG_KEYFRAME = 0x00000008, + BUF_FLAG_PFRAME = 0x00000010, + BUF_FLAG_BFRAME = 0x00000020, + BUF_FLAG_ERROR = 0x00000040, +}; + enum iris_buffer_attributes { BUF_ATTR_DEFERRED = BIT(0), BUF_ATTR_READ_ONLY = BIT(1), diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index 7868566..a5b8aef 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -57,6 +57,18 @@ enum iris_buffer_type v4l2_type_to_driver(u32 type) } } +u32 v4l2_type_from_driver(enum iris_buffer_type buffer_type) +{ + switch (buffer_type) { + case BUF_INPUT: + return INPUT_MPLANE; + case BUF_OUTPUT: + return OUTPUT_MPLANE; + default: + return 0; + } +} + int get_mbpf(struct iris_inst *inst) { int height = 0, width = 0; @@ -390,13 +402,359 @@ static int kill_session(struct iris_inst *inst) return 0; } +struct iris_buffer *get_driver_buf(struct iris_inst *inst, u32 plane, u32 index) +{ + struct iris_buffer *iter = NULL; + struct iris_buffer *buf = NULL; + enum iris_buffer_type buf_type; + struct iris_buffers *buffers; + + bool found = false; + + buf_type = v4l2_type_to_driver(plane); + if (!buf_type) + return NULL; + + buffers = iris_get_buffer_list(inst, buf_type); + if (!buffers) + return NULL; + + list_for_each_entry(iter, &buffers->list, list) { + if (iter->index == index) { + found = true; + buf = iter; + break; + } + } + + if (!found) + return NULL; + + return buf; +} + +static void process_requeued_readonly_buffers(struct iris_inst *inst, + struct iris_buffer *buf) +{ + struct iris_buffer *ro_buf, *dummy; + + list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { + if (ro_buf->device_addr != buf->device_addr) + continue; + if (ro_buf->attr & BUF_ATTR_READ_ONLY && + !(ro_buf->attr & BUF_ATTR_PENDING_RELEASE)) { + buf->attr |= BUF_ATTR_READ_ONLY; + + list_del_init(&ro_buf->list); + kfree(ro_buf); + break; + } + } +} + +int queue_buffer(struct iris_inst *inst, struct iris_buffer *buf) +{ + int ret; + + if (buf->type == BUF_OUTPUT) + process_requeued_readonly_buffers(inst, buf); + + ret = iris_hfi_queue_buffer(inst, buf); + if (ret) + return ret; + + buf->attr &= ~BUF_ATTR_DEFERRED; + buf->attr |= BUF_ATTR_QUEUED; + + return ret; +} + +int queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_type) +{ + struct iris_buffers *buffers; + struct iris_buffer *buf; + int ret = 0; + + buffers = iris_get_buffer_list(inst, buf_type); + if (!buffers) + return -EINVAL; + + list_for_each_entry(buf, &buffers->list, list) { + if (!(buf->attr & BUF_ATTR_DEFERRED)) + continue; + ret = queue_buffer(inst, buf); + if (ret) + return ret; + } + + return ret; +} + +int iris_release_nonref_buffers(struct iris_inst *inst) +{ + u32 fw_ro_count = 0, nonref_ro_count = 0; + struct iris_buffer *ro_buf; + bool found = false; + int ret = 0; + int i = 0; + + list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) { + if (!(ro_buf->attr & BUF_ATTR_READ_ONLY)) + continue; + if (ro_buf->attr & BUF_ATTR_PENDING_RELEASE) + continue; + fw_ro_count++; + } + + if (fw_ro_count <= MAX_DPB_COUNT) + return 0; + + /* + * Mark the read only buffers present in read_only list as + * non-reference if it's not part of dpb_list_payload. + * dpb_list_payload details: + * payload[0-1] : 64 bits base_address of DPB-1 + * payload[2] : 32 bits addr_offset of DPB-1 + * payload[3] : 32 bits data_offset of DPB-1 + */ + list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) { + found = false; + if (!(ro_buf->attr & BUF_ATTR_READ_ONLY)) + continue; + if (ro_buf->attr & BUF_ATTR_PENDING_RELEASE) + continue; + for (i = 0; (i + 3) < MAX_DPB_LIST_ARRAY_SIZE; i = i + 4) { + if (ro_buf->device_addr == inst->dpb_list_payload[i] && + ro_buf->data_offset == inst->dpb_list_payload[i + 3]) { + found = true; + break; + } + } + if (!found) + nonref_ro_count++; + } + + if (nonref_ro_count <= inst->buffers.output.min_count) + return 0; + + list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) { + found = false; + if (!(ro_buf->attr & BUF_ATTR_READ_ONLY)) + continue; + if (ro_buf->attr & BUF_ATTR_PENDING_RELEASE) + continue; + for (i = 0; (i + 3) < MAX_DPB_LIST_ARRAY_SIZE; i = i + 4) { + if (ro_buf->device_addr == inst->dpb_list_payload[i] && + ro_buf->data_offset == inst->dpb_list_payload[i + 3]) { + found = true; + break; + } + } + if (!found) { + ro_buf->attr |= BUF_ATTR_PENDING_RELEASE; + ret = iris_hfi_release_buffer(inst, ro_buf); + if (ret) + return ret; + } + } + + return ret; +} + +int iris_vb2_buffer_done(struct iris_inst *inst, + struct iris_buffer *buf) +{ + struct vb2_v4l2_buffer *vbuf; + struct vb2_queue *q = NULL; + struct vb2_buffer *iter; + struct vb2_buffer *vb2; + int type, state; + bool found; + + type = v4l2_type_from_driver(buf->type); + if (!type) + return -EINVAL; + + if (type == INPUT_MPLANE) + q = inst->vb2q_src; + else if (type == OUTPUT_MPLANE) + q = inst->vb2q_dst; + if (!q || !q->streaming) + return -EINVAL; + + found = false; + list_for_each_entry(iter, &q->queued_list, queued_entry) { + if (iter->state != VB2_BUF_STATE_ACTIVE) + continue; + if (iter->index == buf->index) { + found = true; + vb2 = iter; + break; + } + } + if (!found) + return -EINVAL; + + if (buf->flags & BUF_FLAG_ERROR) + state = VB2_BUF_STATE_ERROR; + else + state = VB2_BUF_STATE_DONE; + + vbuf = to_vb2_v4l2_buffer(vb2); + vbuf->flags = buf->flags; + vb2->timestamp = buf->timestamp; + vb2->planes[0].bytesused = buf->data_size + vb2->planes[0].data_offset; + vb2_buffer_done(vb2, state); + + return 0; +} + +static int iris_flush_deferred_buffers(struct iris_inst *inst, + enum iris_buffer_type type) +{ + struct iris_buffer *buf, *dummy; + struct iris_buffers *buffers; + + buffers = iris_get_buffer_list(inst, type); + if (!buffers) + return -EINVAL; + + list_for_each_entry_safe(buf, dummy, &buffers->list, list) { + if (buf->attr & BUF_ATTR_DEFERRED) { + if (!(buf->attr & BUF_ATTR_BUFFER_DONE)) { + buf->attr |= BUF_ATTR_BUFFER_DONE; + buf->data_size = 0; + iris_vb2_buffer_done(inst, buf); + } + } + } + + return 0; +} + +static int iris_flush_read_only_buffers(struct iris_inst *inst, + enum iris_buffer_type type) +{ + struct iris_buffer *ro_buf, *dummy; + + if (type != BUF_OUTPUT) + return 0; + + list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { + if (ro_buf->attr & BUF_ATTR_READ_ONLY) + continue; + if (ro_buf->attach && ro_buf->sg_table) + dma_buf_unmap_attachment(ro_buf->attach, ro_buf->sg_table, + DMA_BIDIRECTIONAL); + if (ro_buf->attach && ro_buf->dmabuf) + dma_buf_detach(ro_buf->dmabuf, ro_buf->attach); + ro_buf->attach = NULL; + ro_buf->sg_table = NULL; + ro_buf->dmabuf = NULL; + ro_buf->device_addr = 0x0; + list_del_init(&ro_buf->list); + kfree(ro_buf); + } + + return 0; +} + +void iris_destroy_buffers(struct iris_inst *inst) +{ + struct iris_buffer *buf, *dummy; + struct iris_buffers *buffers; + + static const enum iris_buffer_type ext_buf_types[] = { + BUF_INPUT, + BUF_OUTPUT, + }; + static const enum iris_buffer_type internal_buf_types[] = { + BUF_BIN, + BUF_COMV, + BUF_NON_COMV, + BUF_LINE, + BUF_DPB, + BUF_PERSIST, + }; + int i; + + for (i = 0; i < ARRAY_SIZE(internal_buf_types); i++) { + buffers = iris_get_buffer_list(inst, internal_buf_types[i]); + if (!buffers) + continue; + list_for_each_entry_safe(buf, dummy, &buffers->list, list) + iris_destroy_internal_buffer(inst, buf); + } + + list_for_each_entry_safe(buf, dummy, &inst->buffers.read_only.list, list) { + if (buf->attach && buf->sg_table) + dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL); + if (buf->attach && buf->dmabuf) + dma_buf_detach(buf->dmabuf, buf->attach); + list_del_init(&buf->list); + kfree(buf); + } + + for (i = 0; i < ARRAY_SIZE(ext_buf_types); i++) { + buffers = iris_get_buffer_list(inst, ext_buf_types[i]); + if (!buffers) + continue; + list_for_each_entry_safe(buf, dummy, &buffers->list, list) { + if (buf->attach && buf->sg_table) + dma_buf_unmap_attachment(buf->attach, buf->sg_table, + DMA_BIDIRECTIONAL); + if (buf->attach && buf->dmabuf) + dma_buf_detach(buf->dmabuf, buf->attach); + list_del_init(&buf->list); + kfree(buf); + } + } +} + +static int get_num_queued_buffers(struct iris_inst *inst, + enum iris_buffer_type type) +{ + struct iris_buffers *buffers; + struct iris_buffer *vbuf; + int count = 0; + + if (type == BUF_INPUT) + buffers = &inst->buffers.input; + else if (type == BUF_OUTPUT) + buffers = &inst->buffers.output; + else + return count; + + list_for_each_entry(vbuf, &buffers->list, list) { + if (vbuf->type != type) + continue; + if (!(vbuf->attr & BUF_ATTR_QUEUED)) + continue; + count++; + } + + return count; +} + int session_streamoff(struct iris_inst *inst, u32 plane) { enum signal_session_response signal_type; + enum iris_buffer_type buffer_type; u32 hw_response_timeout_val; struct iris_core *core; + int count = 0; int ret; + if (plane == INPUT_MPLANE) { + signal_type = SIGNAL_CMD_STOP_INPUT; + buffer_type = BUF_INPUT; + } else if (plane == OUTPUT_MPLANE) { + signal_type = SIGNAL_CMD_STOP_OUTPUT; + buffer_type = BUF_OUTPUT; + } else { + return -EINVAL; + } + ret = iris_hfi_stop(inst, plane); if (ret) goto error; @@ -417,14 +775,25 @@ int session_streamoff(struct iris_inst *inst, u32 plane) if (ret) goto error; + /* no more queued buffers after streamoff */ + count = get_num_queued_buffers(inst, buffer_type); + if (count) { + ret = -EINVAL; + goto error; + } + ret = iris_inst_state_change_streamoff(inst, plane); if (ret) goto error; + iris_flush_deferred_buffers(inst, buffer_type); + iris_flush_read_only_buffers(inst, buffer_type); return 0; error: kill_session(inst); + iris_flush_deferred_buffers(inst, buffer_type); + iris_flush_read_only_buffers(inst, buffer_type); return ret; } diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index 47a017d..3a9889e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -24,6 +24,7 @@ bool res_is_less_than(u32 width, u32 height, u32 get_port_info(struct iris_inst *inst, enum plat_inst_cap_type cap_id); enum iris_buffer_type v4l2_type_to_driver(u32 type); +u32 v4l2_type_from_driver(enum iris_buffer_type buffer_type); int get_mbpf(struct iris_inst *inst); int close_session(struct iris_inst *inst); @@ -34,7 +35,6 @@ bool is_split_mode_enabled(struct iris_inst *inst); int signal_session_msg_receipt(struct iris_inst *inst, enum signal_session_response cmd); struct iris_inst *to_instance(struct iris_core *core, u32 session_id); -int session_streamoff(struct iris_inst *inst, u32 plane); u32 v4l2_codec_from_driver(struct iris_inst *inst, enum codec_type codec); enum codec_type v4l2_codec_to_driver(struct iris_inst *inst, u32 v4l2_codec); @@ -43,4 +43,13 @@ enum colorformat_type v4l2_colorformat_to_driver(struct iris_inst *inst, u32 v4l struct vb2_queue *get_vb2q(struct iris_inst *inst, u32 type); int check_session_supported(struct iris_inst *inst); +struct iris_buffer *get_driver_buf(struct iris_inst *inst, u32 plane, u32 index); +int queue_buffer(struct iris_inst *inst, struct iris_buffer *buf); +int queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_type); +int iris_vb2_buffer_done(struct iris_inst *inst, + struct iris_buffer *buf); +int iris_release_nonref_buffers(struct iris_inst *inst); +void iris_destroy_buffers(struct iris_inst *inst); +int session_streamoff(struct iris_inst *inst, u32 plane); + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c index b8785a9..8dafd04 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -76,6 +76,34 @@ static u32 hfi_buf_type_from_driver(enum iris_buffer_type buffer_type) } } +u32 hfi_buf_type_to_driver(enum hfi_buffer_type buf_type) +{ + switch (buf_type) { + case HFI_BUFFER_BITSTREAM: + return BUF_INPUT; + case HFI_BUFFER_RAW: + return BUF_OUTPUT; + case HFI_BUFFER_BIN: + return BUF_BIN; + case HFI_BUFFER_ARP: + return BUF_ARP; + case HFI_BUFFER_COMV: + return BUF_COMV; + case HFI_BUFFER_NON_COMV: + return BUF_NON_COMV; + case HFI_BUFFER_LINE: + return BUF_LINE; + case HFI_BUFFER_DPB: + return BUF_DPB; + case HFI_BUFFER_PERSIST: + return BUF_PERSIST; + case HFI_BUFFER_VPSS: + return BUF_VPSS; + default: + return 0; + } +} + u32 get_hfi_codec(struct iris_inst *inst) { switch (inst->codec) { diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h index cf0960b6..8999c28 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -6,6 +6,7 @@ #ifndef _IRIS_HFI_PACKET_H_ #define _IRIS_HFI_PACKET_H_ +#include "hfi_defines.h" #include "iris_core.h" #include "iris_instance.h" @@ -89,6 +90,7 @@ u32 get_hfi_matrix_coefficients(u32 coefficients); u32 get_v4l2_color_primaries(u32 hfi_primaries); u32 get_v4l2_transfer_char(u32 hfi_characterstics); u32 get_v4l2_matrix_coefficients(u32 hfi_coefficients); +u32 hfi_buf_type_to_driver(enum hfi_buffer_type buf_type); int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf); int hfi_packet_sys_init(struct iris_core *core, diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c index 4ca9314..b1236dd 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c @@ -4,6 +4,7 @@ */ #include "hfi_defines.h" +#include "iris_buffer.h" #include "iris_helpers.h" #include "iris_hfi_packet.h" #include "iris_hfi_response.h" @@ -21,6 +22,11 @@ struct iris_inst_hfi_range { int (*handle)(struct iris_inst *inst, struct hfi_packet *pkt); }; +struct iris_hfi_buffer_handle { + enum hfi_buffer_type type; + int (*handle)(struct iris_inst *inst, struct hfi_buffer *buffer); +}; + struct iris_hfi_packet_handle { enum hfi_buffer_type type; int (*handle)(struct iris_inst *inst, struct hfi_packet *pkt); @@ -44,6 +50,94 @@ static void print_sfr_message(struct iris_core *core) } } +static bool is_valid_hfi_buffer_type(u32 buffer_type) +{ + if (buffer_type != HFI_BUFFER_BITSTREAM && + buffer_type != HFI_BUFFER_RAW && + buffer_type != HFI_BUFFER_BIN && + buffer_type != HFI_BUFFER_ARP && + buffer_type != HFI_BUFFER_COMV && + buffer_type != HFI_BUFFER_NON_COMV && + buffer_type != HFI_BUFFER_LINE && + buffer_type != HFI_BUFFER_DPB && + buffer_type != HFI_BUFFER_PERSIST && + buffer_type != HFI_BUFFER_VPSS) { + return false; + } + + return true; +} + +static bool is_valid_hfi_port(u32 port, u32 buffer_type) +{ + if (port == HFI_PORT_NONE && + buffer_type != HFI_BUFFER_PERSIST) + return false; + + if (port != HFI_PORT_BITSTREAM && port != HFI_PORT_RAW) + return false; + + return true; +} + +static int get_driver_buffer_flags(struct iris_inst *inst, u32 hfi_flags) +{ + u32 driver_flags = 0; + + if (inst->hfi_frame_info.picture_type & HFI_PICTURE_IDR) + driver_flags |= BUF_FLAG_KEYFRAME; + else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_P) + driver_flags |= BUF_FLAG_PFRAME; + else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_B) + driver_flags |= BUF_FLAG_BFRAME; + else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_I) + driver_flags |= BUF_FLAG_KEYFRAME; + else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_CRA) + driver_flags |= BUF_FLAG_KEYFRAME; + else if (inst->hfi_frame_info.picture_type & HFI_PICTURE_BLA) + driver_flags |= BUF_FLAG_KEYFRAME; + + if (inst->hfi_frame_info.data_corrupt) + driver_flags |= BUF_FLAG_ERROR; + + if (inst->hfi_frame_info.overflow) + driver_flags |= BUF_FLAG_ERROR; + + return driver_flags; +} + +static bool validate_packet_payload(struct hfi_packet *pkt) +{ + u32 payload_size = 0; + + switch (pkt->payload_info) { + case HFI_PAYLOAD_U32: + case HFI_PAYLOAD_S32: + case HFI_PAYLOAD_Q16: + case HFI_PAYLOAD_U32_ENUM: + case HFI_PAYLOAD_32_PACKED: + payload_size = 4; + break; + case HFI_PAYLOAD_U64: + case HFI_PAYLOAD_S64: + case HFI_PAYLOAD_64_PACKED: + payload_size = 8; + break; + case HFI_PAYLOAD_STRUCTURE: + if (pkt->type == HFI_CMD_BUFFER) + payload_size = sizeof(struct hfi_buffer); + break; + default: + payload_size = 0; + break; + } + + if (pkt->size < sizeof(struct hfi_packet) + payload_size) + return false; + + return true; +} + static int validate_packet(u8 *response_pkt, u8 *core_resp_pkt, u32 core_resp_pkt_size) { u32 response_pkt_size = 0; @@ -168,6 +262,293 @@ static int handle_session_close(struct iris_inst *inst, return 0; } +static int handle_read_only_buffer(struct iris_inst *inst, + struct iris_buffer *buf) +{ + struct iris_buffer *ro_buf, *iter; + bool found = false; + + list_for_each_entry(iter, &inst->buffers.read_only.list, list) { + if (iter->device_addr == buf->device_addr) { + found = true; + ro_buf = iter; + break; + } + } + + if (!found) { + ro_buf = kzalloc(sizeof(*ro_buf), GFP_KERNEL); + if (!ro_buf) + return -ENOMEM; + ro_buf->index = -1; + ro_buf->inst = inst; + ro_buf->type = buf->type; + ro_buf->fd = buf->fd; + ro_buf->dmabuf = buf->dmabuf; + ro_buf->device_addr = buf->device_addr; + ro_buf->data_offset = buf->data_offset; + INIT_LIST_HEAD(&ro_buf->list); + list_add_tail(&ro_buf->list, &inst->buffers.read_only.list); + } + ro_buf->attr |= BUF_ATTR_READ_ONLY; + + return 0; +} + +static int handle_non_read_only_buffer(struct iris_inst *inst, + struct hfi_buffer *buffer) +{ + struct iris_buffer *ro_buf; + + list_for_each_entry(ro_buf, &inst->buffers.read_only.list, list) { + if (ro_buf->device_addr == buffer->base_address) { + ro_buf->attr &= ~BUF_ATTR_READ_ONLY; + break; + } + } + + return 0; +} + +static int handle_release_output_buffer(struct iris_inst *inst, + struct hfi_buffer *buffer) +{ + struct iris_buffer *buf, *iter; + bool found = false; + + list_for_each_entry(iter, &inst->buffers.read_only.list, list) { + if (iter->device_addr == buffer->base_address && + iter->attr & BUF_ATTR_PENDING_RELEASE) { + found = true; + buf = iter; + break; + } + } + if (!found) + return -EINVAL; + + buf->attr &= ~BUF_ATTR_READ_ONLY; + buf->attr &= ~BUF_ATTR_PENDING_RELEASE; + + return 0; +} + +static int handle_input_buffer(struct iris_inst *inst, + struct hfi_buffer *buffer) +{ + struct iris_buffers *buffers; + struct iris_buffer *buf, *iter; + bool found; + + buffers = iris_get_buffer_list(inst, BUF_INPUT); + if (!buffers) + return -EINVAL; + + found = false; + list_for_each_entry(iter, &buffers->list, list) { + if (iter->index == buffer->index) { + found = true; + buf = iter; + break; + } + } + if (!found) + return -EINVAL; + + if (!(buf->attr & BUF_ATTR_QUEUED)) + return 0; + + buf->data_size = buffer->data_size; + buf->attr &= ~BUF_ATTR_QUEUED; + buf->attr |= BUF_ATTR_DEQUEUED; + + buf->flags = get_driver_buffer_flags(inst, buffer->flags); + + return 0; +} + +static int handle_output_buffer(struct iris_inst *inst, + struct hfi_buffer *hfi_buffer) +{ + struct iris_buffers *buffers; + struct iris_buffer *buf, *iter; + bool found; + int ret = 0; + + if (hfi_buffer->flags & HFI_BUF_FW_FLAG_RELEASE_DONE) + return handle_release_output_buffer(inst, hfi_buffer); + + if (!(hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY)) + ret = handle_non_read_only_buffer(inst, hfi_buffer); + + buffers = iris_get_buffer_list(inst, BUF_OUTPUT); + if (!buffers) + return -EINVAL; + + found = false; + list_for_each_entry(iter, &buffers->list, list) { + if (!(iter->attr & BUF_ATTR_QUEUED)) + continue; + + found = (iter->index == hfi_buffer->index && + iter->device_addr == hfi_buffer->base_address && + iter->data_offset == hfi_buffer->data_offset); + + if (found) { + buf = iter; + break; + } + } + if (!found) + return 0; + + buf->data_offset = hfi_buffer->data_offset; + buf->data_size = hfi_buffer->data_size; + buf->timestamp = hfi_buffer->timestamp; + + buf->attr &= ~BUF_ATTR_QUEUED; + buf->attr |= BUF_ATTR_DEQUEUED; + + if (inst->buffers.dpb.size && hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY) + iris_inst_change_state(inst, IRIS_INST_ERROR); + + if (hfi_buffer->flags & HFI_BUF_FW_FLAG_READONLY) { + buf->attr |= BUF_ATTR_READ_ONLY; + ret = handle_read_only_buffer(inst, buf); + } else { + buf->attr &= ~BUF_ATTR_READ_ONLY; + } + + buf->flags = get_driver_buffer_flags(inst, hfi_buffer->flags); + + return ret; +} + +static int handle_dequeue_buffers(struct iris_inst *inst) +{ + struct iris_buffers *buffers; + struct iris_buffer *dummy; + struct iris_buffer *buf; + int ret = 0; + int i; + static const enum iris_buffer_type buffer_type[] = { + BUF_INPUT, + BUF_OUTPUT, + }; + + for (i = 0; i < ARRAY_SIZE(buffer_type); i++) { + buffers = iris_get_buffer_list(inst, buffer_type[i]); + if (!buffers) + return -EINVAL; + + list_for_each_entry_safe(buf, dummy, &buffers->list, list) { + if (buf->attr & BUF_ATTR_DEQUEUED) { + buf->attr &= ~BUF_ATTR_DEQUEUED; + if (!(buf->attr & BUF_ATTR_BUFFER_DONE)) { + buf->attr |= BUF_ATTR_BUFFER_DONE; + ret = iris_vb2_buffer_done(inst, buf); + if (ret) + ret = 0; + } + } + } + } + + return ret; +} + +static int handle_release_internal_buffer(struct iris_inst *inst, + struct hfi_buffer *buffer) +{ + struct iris_buffers *buffers; + struct iris_buffer *buf, *iter; + int ret = 0; + bool found; + + buffers = iris_get_buffer_list(inst, hfi_buf_type_to_driver(buffer->type)); + if (!buffers) + return -EINVAL; + + found = false; + list_for_each_entry(iter, &buffers->list, list) { + if (iter->device_addr == buffer->base_address) { + found = true; + buf = iter; + break; + } + } + if (!found) + return -EINVAL; + + buf->attr &= ~BUF_ATTR_QUEUED; + + if (buf->attr & BUF_ATTR_PENDING_RELEASE) + ret = iris_destroy_internal_buffer(inst, buf); + + return ret; +} + +static int handle_session_buffer(struct iris_inst *inst, + struct hfi_packet *pkt) +{ + struct hfi_buffer *buffer; + u32 hfi_handle_size = 0; + int i, ret = 0; + const struct iris_hfi_buffer_handle *hfi_handle_arr = NULL; + static const struct iris_hfi_buffer_handle input_hfi_handle[] = { + {HFI_BUFFER_BITSTREAM, handle_input_buffer }, + {HFI_BUFFER_BIN, handle_release_internal_buffer }, + {HFI_BUFFER_COMV, handle_release_internal_buffer }, + {HFI_BUFFER_NON_COMV, handle_release_internal_buffer }, + {HFI_BUFFER_LINE, handle_release_internal_buffer }, + {HFI_BUFFER_PERSIST, handle_release_internal_buffer }, + }; + static const struct iris_hfi_buffer_handle output_hfi_handle[] = { + {HFI_BUFFER_RAW, handle_output_buffer }, + {HFI_BUFFER_DPB, handle_release_internal_buffer }, + }; + + if (pkt->payload_info == HFI_PAYLOAD_NONE) + return 0; + + if (!validate_packet_payload(pkt)) { + iris_inst_change_state(inst, IRIS_INST_ERROR); + return 0; + } + + buffer = (struct hfi_buffer *)((u8 *)pkt + sizeof(*pkt)); + if (!is_valid_hfi_buffer_type(buffer->type)) + return 0; + + if (!is_valid_hfi_port(pkt->port, buffer->type)) + return 0; + + if (pkt->port == HFI_PORT_BITSTREAM) { + hfi_handle_size = ARRAY_SIZE(input_hfi_handle); + hfi_handle_arr = input_hfi_handle; + } else if (pkt->port == HFI_PORT_RAW) { + hfi_handle_size = ARRAY_SIZE(output_hfi_handle); + hfi_handle_arr = output_hfi_handle; + } + + if (!hfi_handle_arr || !hfi_handle_size) + return -EINVAL; + + for (i = 0; i < hfi_handle_size; i++) { + if (hfi_handle_arr[i].type == buffer->type) { + ret = hfi_handle_arr[i].handle(inst, buffer); + if (ret) + return ret; + break; + } + } + + if (i == hfi_handle_size) + return -EINVAL; + + return ret; +} + static int handle_src_change(struct iris_inst *inst, struct hfi_packet *pkt) { @@ -191,6 +572,7 @@ static int handle_session_command(struct iris_inst *inst, static const struct iris_hfi_packet_handle hfi_pkt_handle[] = { {HFI_CMD_OPEN, NULL }, {HFI_CMD_CLOSE, handle_session_close }, + {HFI_CMD_BUFFER, handle_session_buffer }, {HFI_CMD_SETTINGS_CHANGE, handle_src_change }, {HFI_CMD_SUBSCRIBE_MODE, NULL }, }; @@ -386,6 +768,7 @@ static int handle_session_response(struct iris_core *core, { struct hfi_packet *packet; struct iris_inst *inst; + bool dequeue = false; u8 *pkt, *start_pkt; int ret = 0; int i, j; @@ -423,6 +806,7 @@ static int handle_session_response(struct iris_core *core, handle_session_error(inst, packet); if (packet->type > be[i].begin && packet->type < be[i].end) { + dequeue |= (packet->type == HFI_CMD_BUFFER); ret = be[i].handle(inst, packet); if (ret) iris_inst_change_state(inst, IRIS_INST_ERROR); @@ -431,7 +815,15 @@ static int handle_session_response(struct iris_core *core, } } + if (dequeue) { + ret = handle_dequeue_buffers(inst); + if (ret) + goto unlock; + } + memset(&inst->hfi_frame_info, 0, sizeof(struct iris_hfi_frame_info)); + +unlock: mutex_unlock(&inst->lock); return ret; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.c b/drivers/media/platform/qcom/vcodec/iris/iris_state.c index 6553029..04e7dc8 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.c @@ -170,6 +170,14 @@ bool allow_reqbufs(struct iris_inst *inst, u32 type) (type == INPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING); } +bool allow_qbuf(struct iris_inst *inst, u32 type) +{ + return (type == INPUT_MPLANE && inst->state == IRIS_INST_INPUT_STREAMING) || + (type == INPUT_MPLANE && inst->state == IRIS_INST_STREAMING) || + (type == OUTPUT_MPLANE && inst->state == IRIS_INST_OUTPUT_STREAMING) || + (type == OUTPUT_MPLANE && inst->state == IRIS_INST_STREAMING); +} + bool allow_streamon(struct iris_inst *inst, u32 type) { return (type == INPUT_MPLANE && inst->state == IRIS_INST_OPEN) || diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_state.h b/drivers/media/platform/qcom/vcodec/iris/iris_state.h index 28d5380..47558ed 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_state.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_state.h @@ -43,6 +43,7 @@ int iris_inst_change_state(struct iris_inst *inst, bool allow_s_fmt(struct iris_inst *inst, u32 type); bool allow_reqbufs(struct iris_inst *inst, u32 type); +bool allow_qbuf(struct iris_inst *inst, u32 type); bool allow_streamon(struct iris_inst *inst, u32 type); bool allow_streamoff(struct iris_inst *inst, u32 type); bool allow_s_ctrl(struct iris_inst *inst, u32 cap_id); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c index d599366..f34434fc 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c @@ -95,6 +95,7 @@ int iris_vb2_queue_setup(struct vb2_queue *q, int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count) { + enum iris_buffer_type buf_type; struct iris_inst *inst; int ret = 0; @@ -132,6 +133,16 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count) if (ret) goto error; + buf_type = v4l2_type_to_driver(q->type); + if (!buf_type) { + ret = -EINVAL; + goto error; + } + + ret = queue_deferred_buffers(inst, buf_type); + if (ret) + goto error; + return ret; error: iris_inst_change_state(inst, IRIS_INST_ERROR); @@ -168,9 +179,33 @@ void iris_vb2_stop_streaming(struct vb2_queue *q) iris_inst_change_state(inst, IRIS_INST_ERROR); } +void iris_vb2_buf_queue(struct vb2_buffer *vb2) +{ + struct iris_inst *inst; + int ret; + + inst = vb2_get_drv_priv(vb2->vb2_queue); + if (!inst || !inst->core) + return; + + if (!vb2->planes[0].bytesused && vb2->type == INPUT_MPLANE) { + ret = -EINVAL; + goto exit; + } + + ret = vdec_qbuf(inst, vb2); + +exit: + if (ret) { + iris_inst_change_state(inst, IRIS_INST_ERROR); + vb2_buffer_done(vb2, VB2_BUF_STATE_ERROR); + } +} + void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev, struct dma_buf *dbuf, unsigned long size) { + struct iris_buffer *ro_buf, *dummy; enum iris_buffer_type buf_type; struct iris_buffers *buffers; struct iris_buffer *iter; @@ -203,6 +238,16 @@ void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev, buf->inst = inst; buf->dmabuf = dbuf; + if (buf->type == BUF_OUTPUT) { + list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { + if (ro_buf->dmabuf != buf->dmabuf) + continue; + buf->attach = ro_buf->attach; + ro_buf->attach = NULL; + return buf; + } + } + buf->attach = dma_buf_attach(dbuf, dev); if (IS_ERR(buf->attach)) { buf->attach = NULL; @@ -214,6 +259,7 @@ void *iris_vb2_attach_dmabuf(struct vb2_buffer *vb, struct device *dev, int iris_vb2_map_dmabuf(void *buf_priv) { + struct iris_buffer *ro_buf, *dummy; struct iris_buffer *buf = buf_priv; struct iris_core *core; struct iris_inst *inst; @@ -229,6 +275,17 @@ int iris_vb2_map_dmabuf(void *buf_priv) return -EINVAL; } + if (buf->type == BUF_OUTPUT) { + list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { + if (ro_buf->dmabuf != buf->dmabuf) + continue; + buf->sg_table = ro_buf->sg_table; + buf->device_addr = ro_buf->device_addr; + ro_buf->sg_table = NULL; + return 0; + } + } + buf->sg_table = dma_buf_map_attachment(buf->attach, DMA_BIDIRECTIONAL); if (IS_ERR(buf->sg_table)) return -EINVAL; @@ -246,6 +303,7 @@ int iris_vb2_map_dmabuf(void *buf_priv) void iris_vb2_unmap_dmabuf(void *buf_priv) { + struct iris_buffer *ro_buf, *dummy; struct iris_buffer *buf = buf_priv; struct iris_core *core; struct iris_inst *inst; @@ -266,6 +324,17 @@ void iris_vb2_unmap_dmabuf(void *buf_priv) return; } + if (buf->type == BUF_OUTPUT) { + list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { + if (ro_buf->dmabuf != buf->dmabuf) + continue; + ro_buf->sg_table = buf->sg_table; + buf->sg_table = NULL; + buf->device_addr = 0x0; + return; + } + } + if (buf->attach && buf->sg_table) { dma_buf_unmap_attachment(buf->attach, buf->sg_table, DMA_BIDIRECTIONAL); buf->sg_table = NULL; @@ -275,6 +344,7 @@ void iris_vb2_unmap_dmabuf(void *buf_priv) void iris_vb2_detach_dmabuf(void *buf_priv) { + struct iris_buffer *ro_buf, *dummy; struct iris_buffer *buf = buf_priv; struct iris_core *core; struct iris_inst *inst; @@ -291,11 +361,22 @@ void iris_vb2_detach_dmabuf(void *buf_priv) buf->sg_table = NULL; } + if (buf->type == BUF_OUTPUT) { + list_for_each_entry_safe(ro_buf, dummy, &inst->buffers.read_only.list, list) { + if (ro_buf->dmabuf != buf->dmabuf) + continue; + ro_buf->attach = buf->attach; + buf->attach = NULL; + goto exit; + } + } + if (buf->attach && buf->dmabuf) { dma_buf_detach(buf->dmabuf, buf->attach); buf->attach = NULL; } +exit: buf->dmabuf = NULL; buf->inst = NULL; } diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h index fc0e804..0757f01 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.h @@ -13,6 +13,7 @@ int iris_vb2_queue_setup(struct vb2_queue *q, unsigned int sizes[], struct device *alloc_devs[]); int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count); void iris_vb2_stop_streaming(struct vb2_queue *q); +void iris_vb2_buf_queue(struct vb2_buffer *vb2); /* vb2_mem_ops */ void *iris_vb2_alloc(struct vb2_buffer *vb, struct device *dev, unsigned long size); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index 0d8ca4b..7eb74c3 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -1193,3 +1193,52 @@ int vdec_streamon_output(struct iris_inst *inst) return ret; } + +static int vb2_buffer_to_driver(struct vb2_buffer *vb2, + struct iris_buffer *buf) +{ + struct vb2_v4l2_buffer *vbuf; + + if (!vb2 || !buf) + return -EINVAL; + + vbuf = to_vb2_v4l2_buffer(vb2); + + buf->fd = vb2->planes[0].m.fd; + buf->data_offset = vb2->planes[0].data_offset; + buf->data_size = vb2->planes[0].bytesused - vb2->planes[0].data_offset; + buf->buffer_size = vb2->planes[0].length; + buf->timestamp = vb2->timestamp; + buf->flags = vbuf->flags; + buf->attr = 0; + + return 0; +} + +int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2) +{ + struct iris_buffer *buf = NULL; + int ret = 0; + + buf = get_driver_buf(inst, vb2->type, vb2->index); + if (!buf) + return -EINVAL; + + ret = vb2_buffer_to_driver(vb2, buf); + if (ret) + return ret; + + if (!allow_qbuf(inst, vb2->type)) { + buf->attr |= BUF_ATTR_DEFERRED; + return 0; + } + + ret = queue_buffer(inst, buf); + if (ret) + return ret; + + if (vb2->type == OUTPUT_MPLANE) + ret = iris_release_nonref_buffers(inst); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h index 0722da1..d666c54 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.h @@ -18,5 +18,6 @@ int vdec_init_src_change_param(struct iris_inst *inst); int vdec_src_change(struct iris_inst *inst); int vdec_streamon_input(struct iris_inst *inst); int vdec_streamon_output(struct iris_inst *inst); +int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c index 14d0077..0165340 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -252,6 +252,7 @@ int vidc_close(struct file *filp) iris_inst_change_state(inst, IRIS_INST_CLOSE); vidc_vb2_queue_deinit(inst); vidc_v4l2_fh_deinit(inst); + iris_destroy_buffers(inst); vidc_remove_session(inst); mutex_unlock(&inst->lock); mutex_destroy(&inst->ctx_q_lock); @@ -912,6 +913,7 @@ static const struct vb2_ops iris_vb2_ops = { .queue_setup = iris_vb2_queue_setup, .start_streaming = iris_vb2_start_streaming, .stop_streaming = iris_vb2_stop_streaming, + .buf_queue = iris_vb2_buf_queue, }; static struct vb2_mem_ops iris_vb2_mem_ops = { From patchwork Mon Dec 18 11:32:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755785 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 345B41CAB2; Mon, 18 Dec 2023 11:40:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="j6MQRq59" Received: from pps.filterd (m0279863.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIBHw62032490; Mon, 18 Dec 2023 11:40:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=VvqHhZOTTYvc/rwKV2zir+K0UrIHVwkgCzFSeUnMfF4=; b=j6 MQRq590+BQX8ZYbrtTHRCXKoEqT1ZmCSXa5c2lg9PYt2gBVuDYEoj3OsbUTinr6H cxWOaDyLiXUfuueqPaJih7mFUlo75OYk4hxZ4igmmoMcXRv3HkSTInrD8UzQOzrf Pk9E7AjZ9AOmp95O4kpTov8Ob0kUmZu+wRAj56jLBR1OqfRpgsykmkXYgplo9wqQ SojX3JyLNgIrJQGBchOg9qxUuvquX0ZGbtDF0w7zOk8IzTaPAnOrqUZEXmNOwsEA TfRwGtewUuOsHaPe8cXvSsjHqha6EKcNFcqaihqMY7aOJleI6WVLNWGQBOOb4Taq WvUfo/tBZy6NE/UQE+eA== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mfe04nr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:03 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBe0Hm004853; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ae-2; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBa9rU000378; Mon, 18 Dec 2023 11:36:09 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX8m5030064; Mon, 18 Dec 2023 11:36:09 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 6725A2383; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 29/34] media: iris: implement power management Date: Mon, 18 Dec 2023 17:02:24 +0530 Message-Id: <1702899149-21321-30-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: bTHmOPnAKerMh8myzEXcM0HtyEWAWpNV X-Proofpoint-GUID: bTHmOPnAKerMh8myzEXcM0HtyEWAWpNV X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 clxscore=1015 suspectscore=0 malwarescore=0 lowpriorityscore=0 mlxscore=0 mlxlogscore=999 phishscore=0 adultscore=0 impostorscore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: From: Vikash Garodia Hardware specific power sequence include programming specific sequence of registers. Implements this sequence for iris3. Introduce video device suspend and resume using runtime PM. Also, implement iris3 specific calculation for clock and bus bandwidth which depends on hardware configuration, codec format, resolution and frame rate. Signed-off-by: Vikash Garodia Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/vcodec/iris/Makefile | 2 + .../media/platform/qcom/vcodec/iris/hfi_defines.h | 7 + .../media/platform/qcom/vcodec/iris/iris_common.h | 7 + .../media/platform/qcom/vcodec/iris/iris_core.h | 4 + .../media/platform/qcom/vcodec/iris/iris_helpers.c | 66 +++ .../media/platform/qcom/vcodec/iris/iris_helpers.h | 4 + drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 175 +++++++- drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 5 + .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 48 +++ .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 4 + .../platform/qcom/vcodec/iris/iris_hfi_response.c | 2 + .../platform/qcom/vcodec/iris/iris_instance.h | 10 + .../media/platform/qcom/vcodec/iris/iris_power.c | 173 ++++++++ .../media/platform/qcom/vcodec/iris/iris_power.h | 15 + .../media/platform/qcom/vcodec/iris/iris_probe.c | 118 +++++- drivers/media/platform/qcom/vcodec/iris/iris_vb2.c | 42 +- .../media/platform/qcom/vcodec/iris/iris_vdec.c | 7 + .../media/platform/qcom/vcodec/iris/iris_vidc.c | 36 +- .../platform/qcom/vcodec/iris/platform_common.h | 18 + .../media/platform/qcom/vcodec/iris/resources.c | 280 +++++++++++++ .../media/platform/qcom/vcodec/iris/resources.h | 8 + .../media/platform/qcom/vcodec/iris/vpu_common.c | 87 ++++ .../media/platform/qcom/vcodec/iris/vpu_common.h | 10 + .../media/platform/qcom/vcodec/iris/vpu_iris3.c | 454 ++++++++++++++++++--- .../platform/qcom/vcodec/iris/vpu_iris3_power.c | 97 +++++ .../platform/qcom/vcodec/iris/vpu_iris3_power.h | 13 + 26 files changed, 1614 insertions(+), 78 deletions(-) create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/iris_power.h create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c create mode 100644 drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h diff --git a/drivers/media/platform/qcom/vcodec/iris/Makefile b/drivers/media/platform/qcom/vcodec/iris/Makefile index 7e3d9f1..4c8f8f6 100644 --- a/drivers/media/platform/qcom/vcodec/iris/Makefile +++ b/drivers/media/platform/qcom/vcodec/iris/Makefile @@ -13,10 +13,12 @@ iris-objs += iris_probe.o \ iris_hfi_response.o \ iris_hfi_packet.o \ iris_buffer.o \ + iris_power.o \ resources.o \ vpu_common.o \ vpu_iris3.o \ vpu_iris3_buffer.o \ + vpu_iris3_power.o \ platform_common.o \ platform_sm8550.o diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index bbe8ca6..872674e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -12,6 +12,7 @@ #define HFI_CMD_BEGIN 0x01000000 #define HFI_CMD_INIT 0x01000001 +#define HFI_CMD_POWER_COLLAPSE 0x01000002 #define HFI_CMD_OPEN 0x01000003 #define HFI_CMD_CLOSE 0x01000004 #define HFI_CMD_START 0x01000005 @@ -46,10 +47,16 @@ enum hfi_property_mode_type { #define HFI_BITMASK_FRAME_MBS_ONLY_FLAG 0x00000001 +#define HFI_TRUE 0x00000001 + +#define HFI_FALSE 0x00000000 + #define HFI_PROP_BEGIN 0x03000000 #define HFI_PROP_IMAGE_VERSION 0x03000001 +#define HFI_PROP_INTRA_FRAME_POWER_COLLAPSE 0x03000002 + enum hfi_codec_type { HFI_CODEC_DECODE_AVC = 1, HFI_CODEC_ENCODE_AVC = 2, diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h index 24b322a..3ab4767 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -28,6 +28,8 @@ #define MAX_DPB_LIST_ARRAY_SIZE (16 * 4) #define MAX_DPB_LIST_PAYLOAD_SIZE (16 * 4 * 4) +#define INPUT_TIMER_LIST_SIZE 30 + enum codec_type { H264 = BIT(0), HEVC = BIT(1), @@ -128,4 +130,9 @@ struct iris_hfi_frame_info { u32 overflow; }; +struct iris_input_timer { + struct list_head list; + u64 time_us; +}; + #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index 6452ea7..50553d2 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -101,6 +101,10 @@ struct iris_core { struct list_head instances; u32 intr_status; char fw_version[IRIS_VERSION_LENGTH]; + struct mutex pm_lock; /* lock for pm operations */ + u32 skip_pc_count; + bool power_enabled; + struct iris_core_power power; }; int iris_core_init(struct iris_core *core); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index a5b8aef..395a189 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -3,12 +3,15 @@ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include + #include "hfi_defines.h" #include "iris_core.h" #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_hfi_packet.h" #include "iris_instance.h" +#include "iris_power.h" #include "iris_vidc.h" int check_core_lock(struct iris_core *core) @@ -479,6 +482,8 @@ int queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_typ if (!buffers) return -EINVAL; + iris_scale_power(inst); + list_for_each_entry(buf, &buffers->list, list) { if (!(buf->attr & BUF_ATTR_DEFERRED)) continue; @@ -775,6 +780,9 @@ int session_streamoff(struct iris_inst *inst, u32 plane) if (ret) goto error; + if (plane == INPUT_MPLANE) + iris_flush_input_timer(inst); + /* no more queued buffers after streamoff */ count = get_num_queued_buffers(inst, buffer_type); if (count) { @@ -797,3 +805,61 @@ int session_streamoff(struct iris_inst *inst, u32 plane) return ret; } + +int iris_pm_get(struct iris_core *core) +{ + int ret; + + mutex_lock(&core->pm_lock); + ret = pm_runtime_resume_and_get(core->dev); + mutex_unlock(&core->pm_lock); + + return ret; +} + +int iris_pm_put(struct iris_core *core, bool autosuspend) +{ + int ret; + + mutex_lock(&core->pm_lock); + + if (autosuspend) + ret = pm_runtime_put_autosuspend(core->dev); + else + ret = pm_runtime_put_sync(core->dev); + if (ret > 0) + ret = 0; + + mutex_unlock(&core->pm_lock); + + return ret; +} + +int iris_pm_get_put(struct iris_core *core) +{ + int ret = 0; + + mutex_lock(&core->pm_lock); + + if (pm_runtime_suspended(core->dev)) { + ret = pm_runtime_resume_and_get(core->dev); + if (ret < 0) + goto error; + + ret = pm_runtime_put_autosuspend(core->dev); + } + if (ret > 0) + ret = 0; + +error: + mutex_unlock(&core->pm_lock); + + return ret; +} + +void iris_pm_touch(struct iris_core *core) +{ + mutex_lock(&core->pm_lock); + pm_runtime_mark_last_busy(core->dev); + mutex_unlock(&core->pm_lock); +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h index 3a9889e..c628b2e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.h @@ -51,5 +51,9 @@ int iris_vb2_buffer_done(struct iris_inst *inst, int iris_release_nonref_buffers(struct iris_inst *inst); void iris_destroy_buffers(struct iris_inst *inst); int session_streamoff(struct iris_inst *inst, u32 plane); +int iris_pm_get(struct iris_core *core); +int iris_pm_put(struct iris_core *core, bool autosuspend); +int iris_pm_get_put(struct iris_core *core); +void iris_pm_touch(struct iris_core *core); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index 2b3fa47..2850fd5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -67,6 +67,41 @@ static bool validate_session(struct iris_core *core, return valid; } +static int iris_power_off(struct iris_core *core) +{ + int ret; + + if (!core->power_enabled) + return 0; + + ret = call_vpu_op(core, power_off, core); + if (ret) { + dev_err(core->dev, "Failed to power off, err: %d\n", ret); + return ret; + } + core->power_enabled = false; + + return ret; +} + +static int iris_power_on(struct iris_core *core) +{ + int ret; + + if (core->power_enabled) + return 0; + + ret = call_vpu_op(core, power_on, core); + if (ret) { + dev_err(core->dev, "Failed to power on, err: %d\n", ret); + return ret; + } + + core->power_enabled = true; + + return ret; +} + static int sys_init(struct iris_core *core) { int ret; @@ -113,20 +148,24 @@ int iris_hfi_core_init(struct iris_core *core) if (ret) goto error; + ret = iris_power_on(core); + if (ret) + goto error; + core->use_tz = use_tz(core->dev); if (!core->use_tz) - goto error; + goto error_power_off; pas_id = core->platform_data->pas_id; fw_name = core->platform_data->fwname; ret = load_fw(core->dev, fw_name, &mem_phys, &mem_size, pas_id, core->use_tz); if (ret) - goto error; + goto error_power_off; ret = auth_reset_fw(pas_id); if (ret) - goto error; + goto error_power_off; cp_start = core->cap[CP_START].value; cp_size = core->cap[CP_SIZE].value; @@ -135,22 +174,24 @@ int iris_hfi_core_init(struct iris_core *core) ret = protect_secure_region(cp_start, cp_size, cp_nonpixel_start, cp_nonpixel_size, pas_id); if (ret) - goto error; + goto error_power_off; ret = call_vpu_op(core, boot_firmware, core); if (ret) - goto error; + goto error_power_off; ret = sys_init(core); if (ret) - goto error; + goto error_power_off; ret = sys_image_version(core); if (ret) - goto error; + goto error_power_off; return ret; +error_power_off: + iris_power_off(core); error: dev_err(core->dev, "%s(): failed with ret %d\n", __func__, ret); @@ -170,6 +211,8 @@ int iris_hfi_core_deinit(struct iris_core *core) unload_fw(core->platform_data->pas_id); + iris_power_off(core); + return ret; } @@ -573,16 +616,24 @@ irqreturn_t iris_hfi_isr(int irq, void *data) irqreturn_t iris_hfi_isr_handler(int irq, void *data) { struct iris_core *core = data; + int ret; if (!core) return IRQ_NONE; + ret = iris_pm_get(core); + if (ret) + goto exit; + mutex_lock(&core->lock); call_vpu_op(core, clear_interrupt, core); mutex_unlock(&core->lock); __response_handler(core); + iris_pm_put(core, true); + +exit: if (!call_vpu_op(core, watchdog, core, core->intr_status)) enable_irq(irq); @@ -701,3 +752,113 @@ int iris_hfi_release_buffer(struct iris_inst *inst, return ret; } + +int prepare_pc(struct iris_core *core) +{ + int ret; + + ret = hfi_packet_sys_pc_prep(core, core->packet, core->packet_size); + if (ret) + goto err_pc_prep; + + ret = iris_hfi_queue_cmd_write(core, core->packet); + if (ret) + goto err_pc_prep; + + return ret; + +err_pc_prep: + dev_err(core->dev, "Failed to prepare iris for power off\n"); + + return ret; +} + +int iris_hfi_pm_suspend(struct iris_core *core) +{ + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + + if (!core_in_valid_state(core)) + return -EINVAL; + + if (!core->power_enabled) + return 0; + + if (core->skip_pc_count >= MAX_PC_SKIP_COUNT) { + dev_err(core->dev, "Failed to PC for %d times\n", core->skip_pc_count); + core->skip_pc_count = 0; + iris_change_core_state(core, IRIS_CORE_ERROR); + iris_core_deinit_locked(core); + return -EINVAL; + } + + iris_flush_debug_queue(core, core->packet, core->packet_size); + + ret = call_vpu_op(core, prepare_pc, core); + if (ret) { + core->skip_pc_count++; + iris_pm_touch(core); + return -EAGAIN; + } + + ret = set_hw_state(false); + if (ret) + return ret; + + ret = iris_power_off(core); + if (ret) + return ret; + + core->skip_pc_count = 0; + + return ret; +} + +int iris_hfi_pm_resume(struct iris_core *core) +{ + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + + if (!core_in_valid_state(core)) + return -EINVAL; + + if (core->power_enabled) + return 0; + + ret = iris_power_on(core); + if (ret) + goto error; + + ret = set_hw_state(true); + if (ret) + goto err_power_off; + + ret = call_vpu_op(core, boot_firmware, core); + if (ret) + goto err_suspend_hw; + + ret = hfi_packet_sys_interframe_powercollapse(core, core->packet, core->packet_size); + if (ret) + goto err_suspend_hw; + + ret = iris_hfi_queue_cmd_write(core, core->packet); + if (ret) + goto err_suspend_hw; + + return ret; + +err_suspend_hw: + set_hw_state(false); +err_power_off: + iris_power_off(core); +error: + dev_err(core->dev, "Failed to Resume\n"); + + return -EBUSY; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h index ab479a9..465bfc5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h @@ -9,6 +9,8 @@ #include "iris_instance.h" #include "iris_core.h" +#define MAX_PC_SKIP_COUNT 10 + int iris_hfi_core_init(struct iris_core *core); int iris_hfi_core_deinit(struct iris_core *core); int iris_hfi_session_open(struct iris_inst *inst); @@ -32,6 +34,9 @@ int iris_hfi_queue_buffer(struct iris_inst *inst, struct iris_buffer *buffer); int iris_hfi_release_buffer(struct iris_inst *inst, struct iris_buffer *buffer); +int iris_hfi_pm_suspend(struct iris_core *core); +int iris_hfi_pm_resume(struct iris_core *core); +int prepare_pc(struct iris_core *core); irqreturn_t iris_hfi_isr(int irq, void *data); irqreturn_t iris_hfi_isr_handler(int irq, void *data); diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c index 8dafd04..fab206cf 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -622,3 +622,51 @@ int hfi_packet_session_property(struct iris_inst *inst, return ret; } + +int hfi_packet_sys_interframe_powercollapse(struct iris_core *core, + u8 *pkt, u32 pkt_size) +{ + u32 payload = 0; + int ret; + + ret = hfi_create_header(pkt, pkt_size, + 0 /*session_id*/, + core->header_id++); + if (ret) + return ret; + + payload = HFI_TRUE; + + ret = hfi_create_packet(pkt, pkt_size, + HFI_PROP_INTRA_FRAME_POWER_COLLAPSE, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); + + return ret; +} + +int hfi_packet_sys_pc_prep(struct iris_core *core, + u8 *pkt, u32 pkt_size) +{ + int ret; + + ret = hfi_create_header(pkt, pkt_size, + 0 /*session_id*/, + core->header_id++); + if (ret) + return ret; + + ret = hfi_create_packet(pkt, pkt_size, + HFI_CMD_POWER_COLLAPSE, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_NONE, + HFI_PORT_NONE, + core->packet_id++, + NULL, 0); + + return ret; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h index 8999c28..849b585 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -104,5 +104,9 @@ int hfi_packet_session_command(struct iris_inst *inst, u32 pkt_type, int hfi_packet_session_property(struct iris_inst *inst, u32 pkt_type, u32 flags, u32 port, u32 payload_type, void *payload, u32 payload_size); +int hfi_packet_sys_interframe_powercollapse(struct iris_core *core, + u8 *pkt, u32 pkt_size); +int hfi_packet_sys_pc_prep(struct iris_core *core, + u8 *pkt, u32 pkt_size); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c index 3548f9d..a4d5d9c 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c @@ -928,6 +928,8 @@ static int handle_response(struct iris_core *core, void *response) struct hfi_header *hdr; int ret; + iris_pm_touch(core); + hdr = (struct hfi_header *)response; ret = validate_hdr_packet(core, hdr); if (ret) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h index 50154bf..3c85e78 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -47,6 +47,11 @@ * @dst_subcr_params: subscription params to fw on output port * @dpb_list_payload: array of dpb buffers * @once_per_session_set: boolean to set once per session property + * @max_rate: max input rate + * @max_input_data_size: max size of input data + * @power: structure of power info + * @bus_data: structure of bus data + * @input_timer_list: list head of input timer */ struct iris_inst { @@ -80,6 +85,11 @@ struct iris_inst { struct subscription_params dst_subcr_params; u32 dpb_list_payload[MAX_DPB_LIST_ARRAY_SIZE]; bool once_per_session_set; + u32 max_rate; + u32 max_input_data_size; + struct iris_inst_power power; + struct bus_vote_data bus_data; + struct list_head input_timer_list; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_power.c b/drivers/media/platform/qcom/vcodec/iris/iris_power.c new file mode 100644 index 0000000..77439e3 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_power.c @@ -0,0 +1,173 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_power.h" +#include "iris_helpers.h" +#include "resources.h" + +static int iris_set_buses(struct iris_inst *inst) +{ + struct iris_inst *instance; + struct iris_core *core; + u64 total_bw_ddr = 0; + int ret; + + core = inst->core; + + mutex_lock(&core->lock); + list_for_each_entry(instance, &core->instances, list) { + if (!instance->max_input_data_size) + continue; + + total_bw_ddr += instance->power.bus_bw; + } + + ret = vote_buses(core, total_bw_ddr); + + mutex_unlock(&core->lock); + + return ret; +} + +static int iris_vote_buses(struct iris_inst *inst) +{ + struct v4l2_format *out_f, *inp_f; + struct bus_vote_data *vote_data; + struct iris_core *core; + + core = inst->core; + + vote_data = &inst->bus_data; + + out_f = inst->fmt_dst; + inp_f = inst->fmt_src; + + vote_data->width = inp_f->fmt.pix_mp.width; + vote_data->height = inp_f->fmt.pix_mp.height; + vote_data->fps = inst->max_rate; + + if (is_linear_colorformat(out_f->fmt.pix_mp.pixelformat)) { + vote_data->color_formats[0] = V4L2_PIX_FMT_NV12; + vote_data->color_formats[1] = out_f->fmt.pix_mp.pixelformat; + } else { + vote_data->color_formats[0] = out_f->fmt.pix_mp.pixelformat; + } + + call_session_op(core, calc_bw, inst, vote_data); + + inst->power.bus_bw = vote_data->bus_bw; + + return iris_set_buses(inst); +} + +static int iris_set_clocks(struct iris_inst *inst) +{ + struct iris_inst *instance; + struct iris_core *core; + int ret = 0; + u64 freq; + + core = inst->core; + + mutex_lock(&core->lock); + + freq = 0; + list_for_each_entry(instance, &core->instances, list) { + if (!instance->max_input_data_size) + continue; + + freq += instance->power.min_freq; + } + + core->power.clk_freq = freq; + + ret = opp_set_rate(core, freq); + + mutex_unlock(&core->lock); + + return ret; +} + +static int iris_scale_clocks(struct iris_inst *inst) +{ + struct iris_buffer *vbuf; + struct iris_core *core; + u32 data_size = 0; + + core = inst->core; + + list_for_each_entry(vbuf, &inst->buffers.input.list, list) + data_size = max(data_size, vbuf->data_size); + + inst->max_input_data_size = data_size; + + inst->max_rate = inst->cap[QUEUED_RATE].value >> 16; + + if (!inst->max_input_data_size) + return 0; + + inst->power.min_freq = call_session_op(core, calc_freq, inst, + inst->max_input_data_size); + iris_set_clocks(inst); + + return 0; +} + +int iris_scale_power(struct iris_inst *inst) +{ + iris_scale_clocks(inst); + iris_vote_buses(inst); + + return 0; +} + +int iris_update_input_rate(struct iris_inst *inst, u64 time_us) +{ + struct iris_input_timer *prev_timer = NULL; + struct iris_input_timer *input_timer; + u64 input_timer_sum_us = 0; + u64 counter = 0; + + input_timer = kzalloc(sizeof(*input_timer), GFP_KERNEL); + if (!input_timer) + return -ENOMEM; + + input_timer->time_us = time_us; + INIT_LIST_HEAD(&input_timer->list); + list_add_tail(&input_timer->list, &inst->input_timer_list); + list_for_each_entry(input_timer, &inst->input_timer_list, list) { + if (prev_timer) { + input_timer_sum_us += input_timer->time_us - prev_timer->time_us; + counter++; + } + prev_timer = input_timer; + } + + if (input_timer_sum_us && counter >= INPUT_TIMER_LIST_SIZE) + inst->cap[QUEUED_RATE].value = + (s32)(DIV64_U64_ROUND_CLOSEST(counter * 1000000, + input_timer_sum_us) << 16); + + if (counter >= INPUT_TIMER_LIST_SIZE) { + input_timer = list_first_entry(&inst->input_timer_list, + struct iris_input_timer, list); + list_del_init(&input_timer->list); + kfree(input_timer); + } + + return 0; +} + +int iris_flush_input_timer(struct iris_inst *inst) +{ + struct iris_input_timer *input_timer, *dummy_timer; + + list_for_each_entry_safe(input_timer, dummy_timer, &inst->input_timer_list, list) { + list_del_init(&input_timer->list); + kfree(input_timer); + } + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_power.h b/drivers/media/platform/qcom/vcodec/iris/iris_power.h new file mode 100644 index 0000000..28dbd5f --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/iris_power.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_POWER_H_ +#define _IRIS_POWER_H_ + +#include "iris_instance.h" + +int iris_scale_power(struct iris_inst *inst); +int iris_update_input_rate(struct iris_inst *inst, u64 time_us); +int iris_flush_input_timer(struct iris_inst *inst); + +#endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index bf484a3..2571e27 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -6,14 +6,15 @@ #include #include #include +#include #include "../hfi_queue.h" #include "iris_core.h" +#include "iris_ctrls.h" #include "iris_helpers.h" #include "iris_hfi.h" -#include "resources.h" #include "iris_vidc.h" -#include "iris_ctrls.h" +#include "resources.h" static int init_iris_isr(struct iris_core *core) { @@ -70,6 +71,8 @@ static void iris_remove(struct platform_device *pdev) if (!core) return; + iris_pm_get(core); + iris_core_deinit(core); hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr, &core->command_queue, &core->message_queue, @@ -79,7 +82,10 @@ static void iris_remove(struct platform_device *pdev) v4l2_device_unregister(&core->v4l2_dev); + iris_pm_put(core, false); + mutex_destroy(&core->lock); + mutex_destroy(&core->pm_lock); core->state = IRIS_CORE_DEINIT; } @@ -97,6 +103,7 @@ static int iris_probe(struct platform_device *pdev) core->state = IRIS_CORE_DEINIT; mutex_init(&core->lock); + mutex_init(&core->pm_lock); core->packet_size = IFACEQ_CORE_PKT_SIZE; core->packet = devm_kzalloc(core->dev, core->packet_size, GFP_KERNEL); @@ -117,58 +124,66 @@ static int iris_probe(struct platform_device *pdev) if (core->irq < 0) return core->irq; + pm_runtime_set_autosuspend_delay(core->dev, AUTOSUSPEND_DELAY_VALUE); + pm_runtime_use_autosuspend(core->dev); + ret = devm_pm_runtime_enable(core->dev); + if (ret) { + dev_err(core->dev, "failed to enable runtime pm\n"); + goto err_runtime_disable; + } + ret = init_iris_isr(core); if (ret) { dev_err_probe(core->dev, ret, "%s: Failed to init isr with %d\n", __func__, ret); - return ret; + goto err_runtime_disable; } ret = init_platform(core); if (ret) { dev_err_probe(core->dev, ret, "%s: init platform failed with %d\n", __func__, ret); - return ret; + goto err_runtime_disable; } ret = init_vpu(core); if (ret) { dev_err_probe(core->dev, ret, "%s: init vpu failed with %d\n", __func__, ret); - return ret; + goto err_runtime_disable; } ret = init_ops(core); if (ret) { dev_err_probe(core->dev, ret, "%s: init ops failed with %d\n", __func__, ret); - return ret; + goto err_runtime_disable; } ret = init_resources(core); if (ret) { dev_err_probe(core->dev, ret, "%s: init resource failed with %d\n", __func__, ret); - return ret; + goto err_runtime_disable; } ret = iris_init_core_caps(core); if (ret) { dev_err_probe(core->dev, ret, "%s: init core caps failed with %d\n", __func__, ret); - return ret; + goto err_runtime_disable; } ret = iris_init_instance_caps(core); if (ret) { dev_err_probe(core->dev, ret, "%s: init inst caps failed with %d\n", __func__, ret); - return ret; + goto err_runtime_disable; } ret = v4l2_device_register(dev, &core->v4l2_dev); if (ret) - return ret; + goto err_runtime_disable; ret = iris_register_video_device(core); if (ret) @@ -194,14 +209,29 @@ static int iris_probe(struct platform_device *pdev) goto err_vdev_unreg; } + ret = iris_pm_get(core); + if (ret) { + dev_err_probe(core->dev, ret, "%s: failed to get runtime pm\n", __func__); + goto err_queue_deinit; + } + ret = iris_core_init(core); if (ret) { dev_err_probe(core->dev, ret, "%s: core init failed\n", __func__); goto err_queue_deinit; } + ret = iris_pm_put(core, false); + if (ret) { + pm_runtime_get_noresume(core->dev); + dev_err_probe(core->dev, ret, "%s: failed to put runtime pm\n", __func__); + goto err_core_deinit; + } + return ret; +err_core_deinit: + iris_core_deinit(core); err_queue_deinit: hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr, &core->command_queue, &core->message_queue, @@ -210,10 +240,77 @@ static int iris_probe(struct platform_device *pdev) video_unregister_device(core->vdev_dec); err_v4l2_unreg: v4l2_device_unregister(&core->v4l2_dev); +err_runtime_disable: + pm_runtime_put_noidle(core->dev); + pm_runtime_set_suspended(core->dev); return ret; } +static int iris_pm_suspend(struct device *dev) +{ + struct iris_core *core; + int ret; + + if (!dev || !dev->driver) + return 0; + + core = dev_get_drvdata(dev); + + mutex_lock(&core->lock); + if (!core_in_valid_state(core)) { + ret = 0; + goto unlock; + } + + if (!core->power_enabled) { + ret = 0; + goto unlock; + } + + ret = iris_hfi_pm_suspend(core); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + +static int iris_pm_resume(struct device *dev) +{ + struct iris_core *core; + int ret; + + if (!dev || !dev->driver) + return 0; + + core = dev_get_drvdata(dev); + + mutex_lock(&core->lock); + if (!core_in_valid_state(core)) { + ret = 0; + goto unlock; + } + + if (core->power_enabled) { + ret = 0; + goto unlock; + } + + ret = iris_hfi_pm_resume(core); + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + +static const struct dev_pm_ops iris_pm_ops = { + SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, + pm_runtime_force_resume) + SET_RUNTIME_PM_OPS(iris_pm_suspend, iris_pm_resume, NULL) +}; + static const struct of_device_id iris_dt_match[] = { { .compatible = "qcom,sm8550-iris", @@ -229,6 +326,7 @@ static struct platform_driver qcom_iris_driver = { .driver = { .name = "qcom-iris", .of_match_table = iris_dt_match, + .pm = &iris_pm_ops, }, }; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c index f34434fc..c0878f1 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vb2.c @@ -10,6 +10,7 @@ #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_instance.h" +#include "iris_power.h" #include "iris_vb2.h" #include "iris_vdec.h" @@ -111,39 +112,52 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count) goto error; } + ret = iris_pm_get(inst->core); + if (ret) + goto error; + if (!inst->once_per_session_set) { inst->once_per_session_set = true; ret = iris_hfi_session_set_codec(inst); if (ret) - goto error; + goto err_pm_get; ret = iris_hfi_session_set_default_header(inst); if (ret) - goto error; + goto err_pm_get; ret = iris_alloc_and_queue_session_int_bufs(inst, BUF_PERSIST); if (ret) - goto error; + goto err_pm_get; } + iris_scale_power(inst); + if (q->type == INPUT_MPLANE) ret = vdec_streamon_input(inst); else if (q->type == OUTPUT_MPLANE) ret = vdec_streamon_output(inst); if (ret) - goto error; + goto err_pm_get; buf_type = v4l2_type_to_driver(q->type); if (!buf_type) { ret = -EINVAL; - goto error; + goto err_pm_get; } ret = queue_deferred_buffers(inst, buf_type); if (ret) + goto err_pm_get; + + ret = iris_pm_put(inst->core, true); + if (ret) goto error; return ret; + +err_pm_get: + iris_pm_put(inst->core, false); error: iris_inst_change_state(inst, IRIS_INST_ERROR); @@ -165,6 +179,10 @@ void iris_vb2_stop_streaming(struct vb2_queue *q) if (q->type != INPUT_MPLANE && q->type != OUTPUT_MPLANE) goto error; + ret = iris_pm_get_put(inst->core); + if (ret) + goto error; + if (q->type == INPUT_MPLANE) ret = session_streamoff(inst, INPUT_MPLANE); else if (q->type == OUTPUT_MPLANE) @@ -181,6 +199,8 @@ void iris_vb2_stop_streaming(struct vb2_queue *q) void iris_vb2_buf_queue(struct vb2_buffer *vb2) { + u64 ktime_ns = ktime_get_ns(); + struct iris_core *core; struct iris_inst *inst; int ret; @@ -188,11 +208,23 @@ void iris_vb2_buf_queue(struct vb2_buffer *vb2) if (!inst || !inst->core) return; + core = inst->core; + if (!vb2->planes[0].bytesused && vb2->type == INPUT_MPLANE) { ret = -EINVAL; goto exit; } + if (vb2->type == INPUT_MPLANE) { + ret = iris_update_input_rate(inst, div_u64(ktime_ns, 1000)); + if (ret) + goto exit; + } + + ret = iris_pm_get_put(core); + if (ret) + goto exit; + ret = vdec_qbuf(inst, vb2); exit: diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c index 7884ba6..371615e 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vdec.c @@ -12,6 +12,7 @@ #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_hfi_packet.h" +#include "iris_power.h" #include "iris_vdec.h" #define UNSPECIFIED_COLOR_FORMAT 5 @@ -1069,6 +1070,8 @@ static int process_streamon_input(struct iris_inst *inst) enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE; int ret; + iris_scale_power(inst); + ret = iris_hfi_start(inst, INPUT_MPLANE); if (ret) return ret; @@ -1150,6 +1153,8 @@ static int process_streamon_output(struct iris_inst *inst) bool drain_pending = false; int ret; + iris_scale_power(inst); + if (inst->sub_state & IRIS_INST_SUB_DRC && inst->sub_state & IRIS_INST_SUB_DRC_LAST) clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST; @@ -1287,6 +1292,8 @@ int vdec_qbuf(struct iris_inst *inst, struct vb2_buffer *vb2) return 0; } + iris_scale_power(inst); + ret = queue_buffer(inst, buf); if (ret) return ret; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c index d37ef04..437d6b4 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_vidc.c @@ -9,12 +9,13 @@ #include "iris_buffer.h" #include "iris_common.h" +#include "iris_ctrls.h" #include "iris_helpers.h" #include "iris_hfi.h" #include "iris_instance.h" +#include "iris_power.h" #include "iris_vdec.h" #include "iris_vidc.h" -#include "iris_ctrls.h" #include "iris_vb2.h" #define VIDC_DRV_NAME "iris_driver" @@ -162,17 +163,23 @@ int vidc_open(struct file *filp) int i = 0; int ret; - ret = iris_core_init(core); + ret = iris_pm_get(core); if (ret) return ret; + ret = iris_core_init(core); + if (ret) + goto fail_pm_put; + ret = iris_core_init_wait(core); if (ret) - return ret; + goto fail_pm_put; inst = kzalloc(sizeof(*inst), GFP_KERNEL); - if (!inst) - return -ENOMEM; + if (!inst) { + ret = -ENOMEM; + goto fail_pm_put; + } inst->core = core; inst->session_id = hash32_ptr(inst); @@ -198,6 +205,7 @@ int vidc_open(struct file *filp) INIT_LIST_HEAD(&inst->buffers.persist.list); INIT_LIST_HEAD(&inst->buffers.vpss.list); INIT_LIST_HEAD(&inst->caps_list); + INIT_LIST_HEAD(&inst->input_timer_list); for (i = 0; i < MAX_SIGNAL; i++) init_completion(&inst->completions[i]); @@ -211,11 +219,16 @@ int vidc_open(struct file *filp) if (ret) goto fail_inst_deinit; + iris_scale_power(inst); + ret = iris_hfi_session_open(inst); if (ret) { dev_err(core->dev, "%s: session open failed\n", __func__); goto fail_core_deinit; } + + iris_pm_put(core, true); + filp->private_data = &inst->fh; return 0; @@ -233,6 +246,8 @@ int vidc_open(struct file *filp) mutex_destroy(&inst->ctx_q_lock); mutex_destroy(&inst->lock); kfree(inst); +fail_pm_put: + iris_pm_put(core, false); return ret; } @@ -240,20 +255,25 @@ int vidc_open(struct file *filp) int vidc_close(struct file *filp) { struct iris_inst *inst; + struct iris_core *core; inst = get_vidc_inst(filp, NULL); if (!inst) return -EINVAL; + core = inst->core; + v4l2_ctrl_handler_free(&inst->ctrl_handler); vdec_inst_deinit(inst); mutex_lock(&inst->lock); + iris_pm_get(core); close_session(inst); iris_inst_change_state(inst, IRIS_INST_CLOSE); vidc_vb2_queue_deinit(inst); vidc_v4l2_fh_deinit(inst); iris_destroy_buffers(inst); vidc_remove_session(inst); + iris_pm_put(core, false); mutex_unlock(&inst->lock); mutex_destroy(&inst->ctx_q_lock); mutex_destroy(&inst->lock); @@ -965,11 +985,17 @@ static int vidc_dec_cmd(struct file *filp, void *fh, goto unlock; } + ret = iris_pm_get(inst->core); + if (ret) + goto unlock; + if (dec->cmd == V4L2_DEC_CMD_START) ret = vdec_start_cmd(inst); else if (dec->cmd == V4L2_DEC_CMD_STOP) ret = vdec_stop_cmd(inst); + iris_pm_put(inst->core, true); + unlock: mutex_unlock(&inst->lock); diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h index 22a8f5b..81de610 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h @@ -15,6 +15,7 @@ struct iris_core; struct iris_inst; #define HW_RESPONSE_TIMEOUT_VALUE (1000) +#define AUTOSUSPEND_DELAY_VALUE (HW_RESPONSE_TIMEOUT_VALUE + 500) #define BIT_DEPTH_8 (8 << 16 | 8) #define BIT_DEPTH_10 (10 << 16 | 10) @@ -64,6 +65,11 @@ struct reg_preset_info { u32 mask; }; +struct iris_core_power { + u64 clk_freq; + u64 bus_bw; +}; + struct ubwc_config_data { u32 max_channels; u32 mal_length; @@ -74,6 +80,18 @@ struct ubwc_config_data { u32 bank_spreading; }; +struct bus_vote_data { + u32 color_formats[2]; + int height, width; + u32 fps; + u64 bus_bw; +}; + +struct iris_inst_power { + u64 min_freq; + u32 bus_bw; +}; + enum plat_core_cap_type { CORE_CAP_NONE = 0, DEC_CODECS, diff --git a/drivers/media/platform/qcom/vcodec/iris/resources.c b/drivers/media/platform/qcom/vcodec/iris/resources.c index 8cfdcd6..c9e3754 100644 --- a/drivers/media/platform/qcom/vcodec/iris/resources.c +++ b/drivers/media/platform/qcom/vcodec/iris/resources.c @@ -4,6 +4,8 @@ */ #include +#include +#include #include #include #include @@ -15,6 +17,8 @@ #include "platform_common.h" #include "resources.h" +#define BW_THRESHOLD 50000 + static void iris_pd_release(void *res) { struct device *pd = (struct device *)res; @@ -59,6 +63,34 @@ static int iris_opp_dl_get(struct device *dev, struct device *supplier) return ret; } +int opp_set_rate(struct iris_core *core, u64 freq) +{ + unsigned long opp_freq = 0; + struct dev_pm_opp *opp; + int ret; + + opp_freq = freq; + + opp = dev_pm_opp_find_freq_ceil(core->dev, &opp_freq); + if (IS_ERR(opp)) { + opp = dev_pm_opp_find_freq_floor(core->dev, &opp_freq); + if (IS_ERR(opp)) { + dev_err(core->dev, + "unable to find freq %lld in opp table\n", freq); + return -EINVAL; + } + } + dev_pm_opp_put(opp); + + ret = dev_pm_opp_set_rate(core->dev, opp_freq); + if (ret) { + dev_err(core->dev, "failed to set rate\n"); + return ret; + } + + return ret; +} + static int init_bus(struct iris_core *core) { const struct bus_info *bus_tbl; @@ -135,6 +167,10 @@ static int init_power_domains(struct iris_core *core) } } + ret = devm_pm_opp_set_clkname(core->dev, "vcodec_core"); + if (ret) + return ret; + ret = devm_pm_opp_of_add_table(core->dev); if (ret) { dev_err(core->dev, "%s: failed to add opp table\n", __func__); @@ -144,6 +180,56 @@ static int init_power_domains(struct iris_core *core) return ret; } +int enable_power_domains(struct iris_core *core, const char *name) +{ + struct power_domain_info *pdinfo = NULL; + int ret; + u32 i; + + ret = opp_set_rate(core, ULONG_MAX); + if (ret) + return ret; + + core->pd_count = core->platform_data->pd_tbl_size; + for (i = 0; i < (core->pd_count - 1); i++) { + pdinfo = &core->power_domain_tbl[i]; + if (strcmp(pdinfo->name, name)) + continue; + ret = pm_runtime_get_sync(pdinfo->genpd_dev); + if (ret < 0) + return ret; + } + + ret = opp_set_rate(core, ULONG_MAX); + if (ret) + return ret; + + return ret; +} + +int disable_power_domains(struct iris_core *core, const char *name) +{ + struct power_domain_info *pdinfo = NULL; + int ret; + u32 i; + + ret = opp_set_rate(core, 0); + if (ret) + return ret; + + core->pd_count = core->platform_data->pd_tbl_size; + for (i = 0; i < (core->pd_count - 1); i++) { + pdinfo = &core->power_domain_tbl[i]; + if (strcmp(pdinfo->name, name)) + continue; + ret = pm_runtime_put_sync(pdinfo->genpd_dev); + if (ret) + return ret; + } + + return ret; +} + static int init_clocks(struct iris_core *core) { const struct clock_info *clk_tbl; @@ -204,6 +290,200 @@ static int init_reset_clocks(struct iris_core *core) return 0; } +int unvote_buses(struct iris_core *core) +{ + struct bus_info *bus = NULL; + int ret = 0; + u32 i; + + core->power.bus_bw = 0; + core->bus_count = core->platform_data->bus_tbl_size; + + for (i = 0; i < core->bus_count; i++) { + bus = &core->bus_tbl[i]; + if (!bus->icc) + return -EINVAL; + + ret = icc_set_bw(bus->icc, 0, 0); + if (ret) + return ret; + } + + return ret; +} + +int vote_buses(struct iris_core *core, unsigned long bus_bw) +{ + unsigned long bw_kbps = 0, bw_prev = 0; + struct bus_info *bus = NULL; + int ret = 0; + u32 i; + + core->bus_count = core->platform_data->bus_tbl_size; + + for (i = 0; i < core->bus_count; i++) { + bus = &core->bus_tbl[i]; + if (bus && bus->icc) { + if (!strcmp(bus->name, "iris-ddr")) { + bw_kbps = bus_bw; + bw_prev = core->power.bus_bw; + } else { + bw_kbps = bus->bw_max_kbps; + bw_prev = core->power.bus_bw ? + bw_kbps : 0; + } + + bw_kbps = clamp_t(typeof(bw_kbps), bw_kbps, + bus->bw_min_kbps, bus->bw_max_kbps); + + if (abs(bw_kbps - bw_prev) < BW_THRESHOLD && bw_prev) + continue; + + ret = icc_set_bw(bus->icc, bw_kbps, 0); + if (ret) + return ret; + + if (!strcmp(bus->name, "iris-ddr")) + core->power.bus_bw = bw_kbps; + } + } + + return ret; +} + +static int deassert_reset_control(struct iris_core *core) +{ + struct reset_info *rcinfo = NULL; + int ret = 0; + u32 i; + + core->reset_count = core->platform_data->clk_rst_tbl_size; + + for (i = 0; i < (core->reset_count - 1); i++) { + rcinfo = &core->reset_tbl[i]; + ret = reset_control_deassert(rcinfo->rst); + if (ret) { + dev_err(core->dev, "deassert reset control failed. ret = %d\n", ret); + continue; + } + } + + return ret; +} + +static int assert_reset_control(struct iris_core *core) +{ + struct reset_info *rcinfo = NULL; + int ret = 0, cnt = 0; + u32 i; + + core->reset_count = core->platform_data->clk_rst_tbl_size; + + for (i = 0; i < (core->reset_count - 1); i++) { + rcinfo = &core->reset_tbl[i]; + if (!rcinfo->rst) + return -EINVAL; + + ret = reset_control_assert(rcinfo->rst); + if (ret) { + dev_err(core->dev, "failed to assert reset control %s, ret = %d\n", + rcinfo->name, ret); + goto deassert_reset_control; + } + cnt++; + + usleep_range(1000, 1100); + } + + return ret; +deassert_reset_control: + for (i = 0; i < cnt; i++) { + rcinfo = &core->reset_tbl[i]; + reset_control_deassert(rcinfo->rst); + } + + return ret; +} + +int reset_ahb2axi_bridge(struct iris_core *core) +{ + int ret; + + ret = assert_reset_control(core); + if (ret) + return ret; + + ret = deassert_reset_control(core); + + return ret; +} + +int disable_unprepare_clock(struct iris_core *core, const char *clk_name) +{ + struct clock_info *cl; + bool found = false; + u32 i; + + core->clk_count = core->platform_data->clk_tbl_size; + + for (i = 0; i < core->clk_count; i++) { + cl = &core->clock_tbl[i]; + if (!cl->clk) + return -EINVAL; + + if (strcmp(cl->name, clk_name)) + continue; + + found = true; + clk_disable_unprepare(cl->clk); + cl->prev = 0; + break; + } + + if (!found) + return -EINVAL; + + return 0; +} + +int prepare_enable_clock(struct iris_core *core, const char *clk_name) +{ + struct clock_info *cl; + bool found = false; + int ret = 0; + u32 i; + + core->clk_count = core->platform_data->clk_tbl_size; + + for (i = 0; i < core->clk_count; i++) { + cl = &core->clock_tbl[i]; + if (!cl->clk) + return -EINVAL; + + if (strcmp(cl->name, clk_name)) + continue; + + found = true; + + ret = clk_prepare_enable(cl->clk); + if (ret) { + dev_err(core->dev, "failed to enable clock %s\n", cl->name); + return ret; + } + + if (!__clk_is_enabled(cl->clk)) { + clk_disable_unprepare(cl->clk); + return -EINVAL; + } + break; + } + + if (!found) + return -EINVAL; + + return ret; +} + int init_resources(struct iris_core *core) { int ret; diff --git a/drivers/media/platform/qcom/vcodec/iris/resources.h b/drivers/media/platform/qcom/vcodec/iris/resources.h index d21bcc7e..09857ed 100644 --- a/drivers/media/platform/qcom/vcodec/iris/resources.h +++ b/drivers/media/platform/qcom/vcodec/iris/resources.h @@ -31,6 +31,14 @@ struct reset_info { const char *name; }; +int enable_power_domains(struct iris_core *core, const char *name); +int disable_power_domains(struct iris_core *core, const char *name); +int unvote_buses(struct iris_core *core); +int vote_buses(struct iris_core *core, unsigned long bus_bw); +int reset_ahb2axi_bridge(struct iris_core *core); +int opp_set_rate(struct iris_core *core, u64 freq); +int disable_unprepare_clock(struct iris_core *core, const char *clk_name); +int prepare_enable_clock(struct iris_core *core, const char *clk_name); int init_resources(struct iris_core *core); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.c b/drivers/media/platform/qcom/vcodec/iris/vpu_common.c index 3282510..8ef1029 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_common.c +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.c @@ -3,6 +3,8 @@ * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include + #include "vpu_iris3.h" #include "iris_core.h" #include "iris_helpers.h" @@ -17,6 +19,9 @@ int write_register(struct iris_core *core, u32 reg, u32 value) if (ret) return ret; + if (!core->power_enabled) + return -EINVAL; + base_addr = core->reg_base; base_addr += reg; writel_relaxed(value, base_addr); @@ -27,10 +32,46 @@ int write_register(struct iris_core *core, u32 reg, u32 value) return ret; } +int write_register_masked(struct iris_core *core, u32 reg, u32 value, u32 mask) +{ + void __iomem *base_addr; + u32 prev_val, new_val; + int ret; + + ret = check_core_lock(core); + if (ret) + return ret; + + if (!core->power_enabled) + return -EINVAL; + + base_addr = core->reg_base; + base_addr += reg; + + prev_val = readl_relaxed(base_addr); + /* + * Memory barrier to ensure register read is correct + */ + rmb(); + + new_val = (prev_val & ~mask) | (value & mask); + + writel_relaxed(new_val, base_addr); + /* + * Memory barrier to make sure value is written into the register. + */ + wmb(); + + return ret; +} + int read_register(struct iris_core *core, u32 reg, u32 *value) { void __iomem *base_addr; + if (!core->power_enabled) + return -EINVAL; + base_addr = core->reg_base; *value = readl_relaxed(base_addr + reg); @@ -41,6 +82,52 @@ int read_register(struct iris_core *core, u32 reg, u32 *value) return 0; } +int read_register_with_poll_timeout(struct iris_core *core, u32 reg, + u32 mask, u32 exp_val, u32 sleep_us, + u32 timeout_us) +{ + void __iomem *base_addr; + u32 val = 0; + int ret; + + if (!core->power_enabled) + return -EINVAL; + + base_addr = core->reg_base; + + ret = readl_relaxed_poll_timeout(base_addr + reg, val, ((val & mask) == exp_val), + sleep_us, timeout_us); + /* + * Memory barrier to make sure value is read correctly from the + * register. + */ + rmb(); + + return ret; +} + +int set_preset_registers(struct iris_core *core) +{ + const struct reg_preset_info *reg_prst; + unsigned int prst_count; + int cnt, ret = 0; + + reg_prst = core->platform_data->reg_prst_tbl; + prst_count = core->platform_data->reg_prst_tbl_size; + + if (!reg_prst || !prst_count) + return 0; + + for (cnt = 0; cnt < prst_count; cnt++) { + ret = write_register_masked(core, reg_prst[cnt].reg, + reg_prst[cnt].value, reg_prst[cnt].mask); + if (ret) + return ret; + } + + return ret; +} + static const struct compat_handle compat_handle[] = { { .compat = "qcom,sm8550-iris", diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h index 7fba8c2..aef5606 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_common.h @@ -24,6 +24,9 @@ struct vpu_ops { int (*raise_interrupt)(struct iris_core *core); int (*clear_interrupt)(struct iris_core *core); int (*watchdog)(struct iris_core *core, u32 intr_status); + int (*power_on)(struct iris_core *core); + int (*power_off)(struct iris_core *core); + int (*prepare_pc)(struct iris_core *core); }; #define call_session_op(c, op, ...) \ @@ -32,11 +35,18 @@ struct vpu_ops { struct vpu_session_ops { int (*int_buf_size)(struct iris_inst *inst, enum iris_buffer_type type); + u64 (*calc_freq)(struct iris_inst *inst, u32 data_size); + int (*calc_bw)(struct iris_inst *inst, struct bus_vote_data *data); }; int init_vpu(struct iris_core *core); int write_register(struct iris_core *core, u32 reg, u32 value); +int write_register_masked(struct iris_core *core, u32 reg, u32 value, u32 mask); int read_register(struct iris_core *core, u32 reg, u32 *value); +int read_register_with_poll_timeout(struct iris_core *core, u32 reg, + u32 mask, u32 exp_val, u32 sleep_us, + u32 timeout_us); +int set_preset_registers(struct iris_core *core); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c index efea5aa..85ed38d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3.c @@ -5,73 +5,105 @@ #include +#include "iris_hfi.h" #include "vpu_iris3.h" #include "vpu_iris3_buffer.h" +#include "vpu_iris3_power.h" #define VIDEO_ARCH_LX 1 -#define CPU_BASE_OFFS_IRIS3 0x000A0000 - -#define CPU_CS_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3) -#define CPU_IC_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3) - -#define CPU_CS_VCICMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x24) -#define CPU_CS_VCICMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x28) - -#define CPU_CS_A2HSOFTINTCLR_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x1C) +#define VCODEC_BASE_OFFS_IRIS3 0x00000000 +#define AON_MVP_NOC_RESET 0x0001F000 +#define CPU_BASE_OFFS_IRIS3 0x000A0000 +#define AON_BASE_OFFS 0x000E0000 +#define CPU_CS_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3) +#define CPU_IC_BASE_OFFS_IRIS3 (CPU_BASE_OFFS_IRIS3) +#define CPU_CS_A2HSOFTINTCLR_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x1C) +#define CPU_CS_VCICMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x24) +#define CPU_CS_VCICMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x28) /* HFI_CTRL_INIT */ -#define CPU_CS_SCIACMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x48) - +#define CPU_CS_SCIACMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x48) /* HFI_CTRL_STATUS */ -#define CPU_CS_SCIACMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x4C) -#define CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3 0x40000000 - -#define CPU_CS_H2XSOFTINTEN_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x148) - -#define CPU_CS_X2RPMH_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x168) - -/* UC_REGION_ADDR */ -#define CPU_CS_SCIBARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x64) - -/* UC_REGION_ADDR */ -#define CPU_CS_SCIBARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x68) - +#define CPU_CS_SCIACMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x4C) /* HFI_QTBL_INFO */ -#define CPU_CS_SCIACMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x50) - +#define CPU_CS_SCIACMDARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x50) /* HFI_QTBL_ADDR */ -#define CPU_CS_SCIACMDARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x54) - +#define CPU_CS_SCIACMDARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x54) /* SFR_ADDR */ -#define CPU_CS_SCIBCMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x5C) - -#define UC_REGION_ADDR_IRIS3 CPU_CS_SCIBARG1_IRIS3 -#define UC_REGION_SIZE_IRIS3 CPU_CS_SCIBARG2_IRIS3 - -#define QTBL_INFO_IRIS3 CPU_CS_SCIACMDARG1_IRIS3 -#define QTBL_ADDR_IRIS3 CPU_CS_SCIACMDARG2_IRIS3 - -#define SFR_ADDR_IRIS3 CPU_CS_SCIBCMD_IRIS3 +#define CPU_CS_SCIBCMD_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x5C) +#define CPU_CS_SCIBCMDARG0_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x60) +/* UC_REGION_ADDR */ +#define CPU_CS_SCIBARG1_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x64) +/* UC_REGION_ADDR */ +#define CPU_CS_SCIBARG2_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x68) +#define CPU_CS_H2XSOFTINTEN_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x148) +#define CPU_CS_AHB_BRIDGE_SYNC_RESET (CPU_CS_BASE_OFFS_IRIS3 + 0x160) +#define CPU_CS_X2RPMH_IRIS3 (CPU_CS_BASE_OFFS_IRIS3 + 0x168) -#define CTRL_INIT_IRIS3 CPU_CS_SCIACMD_IRIS3 +#define CPU_IC_SOFTINT_IRIS3 (CPU_IC_BASE_OFFS_IRIS3 + 0x150) +#define CPU_IC_SOFTINT_H2A_SHFT_IRIS3 0x0 -#define CTRL_STATUS_IRIS3 CPU_CS_SCIACMDARG0_IRIS3 -#define CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 0xfe -#define CTRL_ERROR_STATUS__M_IRIS3 \ - CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 -#define CTRL_INIT_IDLE_MSG_BMSK_IRIS3 \ - CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3 +#define CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3 0x40000000 +#define CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 0xfe +#define CPU_CS_SCIACMDARG0_HFI_CTRL_PC_READY_IRIS3 0x100 + +#define AON_WRAPPER_MVP_NOC_RESET_REQ (AON_MVP_NOC_RESET + 0x000) +#define AON_WRAPPER_MVP_NOC_RESET_ACK (AON_MVP_NOC_RESET + 0x004) + +#define WRAPPER_BASE_OFFS_IRIS3 0x000B0000 +#define WRAPPER_INTR_STATUS_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x0C) +#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8 +#define WRAPPER_INTR_STATUS_A2H_BMSK_IRIS3 0x4 + +#define WRAPPER_INTR_MASK_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x10) +#define WRAPPER_INTR_MASK_A2HWD_BMSK_IRIS3 0x8 +#define WRAPPER_INTR_MASK_A2HCPU_BMSK_IRIS3 0x4 + +#define WRAPPER_DEBUG_BRIDGE_LPI_CONTROL_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x54) +#define WRAPPER_DEBUG_BRIDGE_LPI_STATUS_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x58) +#define WRAPPER_IRIS_CPU_NOC_LPI_CONTROL (WRAPPER_BASE_OFFS_IRIS3 + 0x5C) +#define WRAPPER_IRIS_CPU_NOC_LPI_STATUS (WRAPPER_BASE_OFFS_IRIS3 + 0x60) +#define WRAPPER_CORE_POWER_STATUS (WRAPPER_BASE_OFFS_IRIS3 + 0x80) +#define WRAPPER_CORE_CLOCK_CONFIG_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x88) + +#define WRAPPER_TZ_BASE_OFFS 0x000C0000 +#define WRAPPER_TZ_CPU_STATUS (WRAPPER_TZ_BASE_OFFS + 0x10) +#define WRAPPER_TZ_CTL_AXI_CLOCK_CONFIG (WRAPPER_TZ_BASE_OFFS + 0x14) +#define WRAPPER_TZ_QNS4PDXFIFO_RESET (WRAPPER_TZ_BASE_OFFS + 0x18) + +#define CTRL_INIT_IRIS3 CPU_CS_SCIACMD_IRIS3 +#define CTRL_STATUS_IRIS3 CPU_CS_SCIACMDARG0_IRIS3 +#define CTRL_ERROR_STATUS__M_IRIS3 CPU_CS_SCIACMDARG0_HFI_CTRL_ERROR_STATUS_BMSK_IRIS3 +#define CTRL_INIT_IDLE_MSG_BMSK_IRIS3 CPU_CS_SCIACMDARG0_HFI_CTRL_INIT_IDLE_MSG_BMSK_IRIS3 +#define CTRL_STATUS_PC_READY_IRIS3 CPU_CS_SCIACMDARG0_HFI_CTRL_PC_READY_IRIS3 + +#define QTBL_INFO_IRIS3 CPU_CS_SCIACMDARG1_IRIS3 +#define QTBL_ADDR_IRIS3 CPU_CS_SCIACMDARG2_IRIS3 +#define SFR_ADDR_IRIS3 CPU_CS_SCIBCMD_IRIS3 +#define UC_REGION_ADDR_IRIS3 CPU_CS_SCIBARG1_IRIS3 +#define UC_REGION_SIZE_IRIS3 CPU_CS_SCIBARG2_IRIS3 + +#define AON_WRAPPER_MVP_NOC_LPI_CONTROL (AON_BASE_OFFS) +#define AON_WRAPPER_MVP_NOC_LPI_STATUS (AON_BASE_OFFS + 0x4) + +#define VCODEC_SS_IDLE_STATUSN (VCODEC_BASE_OFFS_IRIS3 + 0x70) + +static int interrupt_init_iris3(struct iris_core *core) +{ + u32 mask_val; + int ret; -#define WRAPPER_BASE_OFFS_IRIS3 0x000B0000 -#define WRAPPER_INTR_STATUS_IRIS3 (WRAPPER_BASE_OFFS_IRIS3 + 0x0C) -#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8 -#define WRAPPER_INTR_STATUS_A2H_BMSK_IRIS3 0x4 + ret = read_register(core, WRAPPER_INTR_MASK_IRIS3, &mask_val); + if (ret) + return ret; -#define CPU_IC_SOFTINT_IRIS3 (CPU_IC_BASE_OFFS_IRIS3 + 0x150) -#define CPU_IC_SOFTINT_H2A_SHFT_IRIS3 0x0 + mask_val &= ~(WRAPPER_INTR_MASK_A2HWD_BMSK_IRIS3 | + WRAPPER_INTR_MASK_A2HCPU_BMSK_IRIS3); + ret = write_register(core, WRAPPER_INTR_MASK_IRIS3, mask_val); -#define WRAPPER_INTR_STATUS_A2HWD_BMSK_IRIS3 0x8 + return ret; +} static int setup_ucregion_memory_map_iris3(struct iris_core *core) { @@ -196,15 +228,335 @@ static int watchdog_iris3(struct iris_core *core, u32 intr_status) return 0; } +static bool is_iris3_hw_power_collapsed(struct iris_core *core) +{ + u32 value = 0, pwr_status = 0; + int ret; + + ret = read_register(core, WRAPPER_CORE_POWER_STATUS, &value); + if (ret) + return false; + + pwr_status = value & BIT(1); + + return pwr_status ? false : true; +} + +static int power_off_iris3_hardware(struct iris_core *core) +{ + u32 value = 0; + int ret, i; + + if (is_iris3_hw_power_collapsed(core)) + goto disable_power; + + dev_err(core->dev, "Video hw is power ON\n"); + + ret = read_register(core, WRAPPER_CORE_CLOCK_CONFIG_IRIS3, &value); + if (ret) + goto disable_power; + + if (value) { + ret = write_register(core, WRAPPER_CORE_CLOCK_CONFIG_IRIS3, 0); + if (ret) + goto disable_power; + } + + for (i = 0; i < core->cap[NUM_VPP_PIPE].value; i++) { + ret = read_register_with_poll_timeout(core, VCODEC_SS_IDLE_STATUSN + 4 * i, + 0x400000, 0x400000, 2000, 20000); + } + + ret = write_register(core, AON_WRAPPER_MVP_NOC_RESET_REQ, 0x3); + if (ret) + goto disable_power; + + ret = read_register_with_poll_timeout(core, AON_WRAPPER_MVP_NOC_RESET_ACK, + 0x3, 0x3, 200, 2000); + ret = write_register(core, AON_WRAPPER_MVP_NOC_RESET_REQ, 0x0); + if (ret) + goto disable_power; + + ret = read_register_with_poll_timeout(core, AON_WRAPPER_MVP_NOC_RESET_ACK, + 0x3, 0x0, 200, 2000); + + ret = write_register(core, CPU_CS_AHB_BRIDGE_SYNC_RESET, 0x3); + if (ret) + goto disable_power; + ret = write_register(core, CPU_CS_AHB_BRIDGE_SYNC_RESET, 0x2); + if (ret) + goto disable_power; + ret = write_register(core, CPU_CS_AHB_BRIDGE_SYNC_RESET, 0x0); + if (ret) + goto disable_power; + +disable_power: + ret = disable_power_domains(core, "vcodec"); + if (ret) { + dev_err(core->dev, "disable power domain vcodec failed\n"); + ret = 0; + } + + disable_unprepare_clock(core, "vcodec_core"); + if (ret) { + dev_err(core->dev, "disable unprepare vcodec_core failed\n"); + ret = 0; + } + + return ret; +} + +static int power_off_iris3_controller(struct iris_core *core) +{ + int ret; + + ret = write_register(core, CPU_CS_X2RPMH_IRIS3, 0x3); + if (ret) + goto disable_power; + + ret = write_register_masked(core, AON_WRAPPER_MVP_NOC_LPI_CONTROL, + 0x1, BIT(0)); + if (ret) + goto disable_power; + + ret = read_register_with_poll_timeout(core, AON_WRAPPER_MVP_NOC_LPI_STATUS, + 0x1, 0x1, 200, 2000); + + ret = write_register_masked(core, WRAPPER_IRIS_CPU_NOC_LPI_CONTROL, + 0x1, BIT(0)); + if (ret) + goto disable_power; + + ret = read_register_with_poll_timeout(core, WRAPPER_IRIS_CPU_NOC_LPI_STATUS, + 0x1, 0x1, 200, 2000); + + ret = write_register(core, WRAPPER_DEBUG_BRIDGE_LPI_CONTROL_IRIS3, 0x0); + if (ret) + goto disable_power; + + ret = read_register_with_poll_timeout(core, WRAPPER_DEBUG_BRIDGE_LPI_STATUS_IRIS3, + 0xffffffff, 0x0, 200, 2000); + + ret = write_register(core, WRAPPER_TZ_CTL_AXI_CLOCK_CONFIG, 0x3); + if (ret) + goto disable_power; + + ret = write_register(core, WRAPPER_TZ_QNS4PDXFIFO_RESET, 0x1); + if (ret) + goto disable_power; + + ret = write_register(core, WRAPPER_TZ_QNS4PDXFIFO_RESET, 0x0); + if (ret) + goto disable_power; + + ret = write_register(core, WRAPPER_TZ_CTL_AXI_CLOCK_CONFIG, 0x0); + if (ret) + goto disable_power; + +disable_power: + disable_unprepare_clock(core, "core_clk"); + if (ret) { + dev_err(core->dev, "disable unprepare core_clk failed\n"); + ret = 0; + } + + /* power down process */ + ret = disable_power_domains(core, "iris-ctl"); + if (ret) { + dev_err(core->dev, "disable power domain iris-ctl failed\n"); + ret = 0; + } + + return ret; +} + +static int power_off_iris3(struct iris_core *core) +{ + if (!core->power_enabled) + return 0; + + opp_set_rate(core, 0); + power_off_iris3_hardware(core); + power_off_iris3_controller(core); + unvote_buses(core); + + if (!call_vpu_op(core, watchdog, core, core->intr_status)) + disable_irq_nosync(core->irq); + + core->power_enabled = false; + + return 0; +} + +static int power_on_iris3_controller(struct iris_core *core) +{ + int ret; + + ret = enable_power_domains(core, "iris-ctl"); + if (ret) + return ret; + + ret = reset_ahb2axi_bridge(core); + if (ret) + goto err_disable_power; + + ret = prepare_enable_clock(core, "gcc_video_axi0"); + if (ret) + goto err_disable_power; + + ret = prepare_enable_clock(core, "core_clk"); + if (ret) + goto err_disable_clock; + + return ret; + +err_disable_clock: + disable_unprepare_clock(core, "gcc_video_axi0"); +err_disable_power: + disable_power_domains(core, "iris-ctl"); + + return ret; +} + +static int power_on_iris3_hardware(struct iris_core *core) +{ + int ret; + + ret = enable_power_domains(core, "vcodec"); + if (ret) + return ret; + + ret = prepare_enable_clock(core, "vcodec_core"); + if (ret) + goto err_disable_power; + + return ret; + +err_disable_power: + disable_power_domains(core, "vcodec"); + + return ret; +} + +static int power_on_iris3(struct iris_core *core) +{ + u32 freq = 0; + int ret; + + if (core->power_enabled) + return 0; + + if (!core_in_valid_state(core)) + return -EINVAL; + + ret = vote_buses(core, INT_MAX); + if (ret) + goto err; + + ret = power_on_iris3_controller(core); + if (ret) + goto err_unvote_bus; + + ret = power_on_iris3_hardware(core); + if (ret) + goto err_power_off_ctrl; + + core->power_enabled = true; + + freq = core->power.clk_freq ? core->power.clk_freq : + (u32)ULONG_MAX; + + opp_set_rate(core, freq); + + set_preset_registers(core); + + interrupt_init_iris3(core); + core->intr_status = 0; + enable_irq(core->irq); + + return ret; + +err_power_off_ctrl: + power_off_iris3_controller(core); +err_unvote_bus: + unvote_buses(core); +err: + core->power_enabled = false; + + return ret; +} + +static int prepare_pc_iris3(struct iris_core *core) +{ + u32 wfi_status = 0, idle_status = 0, pc_ready = 0; + u32 ctrl_status = 0; + int ret; + + ret = read_register(core, CTRL_STATUS_IRIS3, &ctrl_status); + if (ret) + return ret; + + pc_ready = ctrl_status & CTRL_STATUS_PC_READY_IRIS3; + idle_status = ctrl_status & BIT(30); + + if (pc_ready) + return 0; + + ret = read_register(core, WRAPPER_TZ_CPU_STATUS, &wfi_status); + if (ret) + return ret; + + wfi_status &= BIT(0); + if (!wfi_status || !idle_status) + goto skip_power_off; + + ret = prepare_pc(core); + if (ret) + goto skip_power_off; + + ret = read_register_with_poll_timeout(core, CTRL_STATUS_IRIS3, + CTRL_STATUS_PC_READY_IRIS3, + CTRL_STATUS_PC_READY_IRIS3, 250, 2500); + if (ret) + goto skip_power_off; + + ret = read_register_with_poll_timeout(core, WRAPPER_TZ_CPU_STATUS, + BIT(0), 0x1, 250, 2500); + if (ret) + goto skip_power_off; + + return ret; + +skip_power_off: + ret = read_register(core, CTRL_STATUS_IRIS3, &ctrl_status); + if (ret) + return ret; + + ret = read_register(core, WRAPPER_TZ_CPU_STATUS, &wfi_status); + if (ret) + return ret; + + wfi_status &= BIT(0); + dev_err(core->dev, "Skip PC, wfi=%#x, idle=%#x, pcr=%#x, ctrl=%#x)\n", + wfi_status, idle_status, pc_ready, ctrl_status); + + return -EAGAIN; +} + static const struct vpu_ops iris3_ops = { .boot_firmware = boot_firmware_iris3, .raise_interrupt = raise_interrupt_iris3, .clear_interrupt = clear_interrupt_iris3, .watchdog = watchdog_iris3, + .power_on = power_on_iris3, + .power_off = power_off_iris3, + .prepare_pc = prepare_pc_iris3, }; static const struct vpu_session_ops iris3_session_ops = { .int_buf_size = iris_int_buf_size_iris3, + .calc_freq = iris_calc_freq_iris3, + .calc_bw = iris_calc_bw_iris3, }; int init_iris3(struct iris_core *core) diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c new file mode 100644 index 0000000..5a02f24 --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c @@ -0,0 +1,97 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_core.h" +#include "iris_instance.h" +#include "iris_helpers.h" +#include "platform_common.h" +#include "vpu_iris3_power.h" + +u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size) +{ + u64 vsp_cycles = 0, vpp_cycles = 0, fw_cycles = 0; + u64 fw_vpp_cycles = 0, bitrate = 0, freq = 0; + u32 base_cycles = 0, fps, mbpf; + u32 height = 0, width = 0; + struct v4l2_format *inp_f; + u32 mbs_per_second; + + inp_f = inst->fmt_src; + width = max(inp_f->fmt.pix_mp.width, inst->crop.width); + height = max(inp_f->fmt.pix_mp.height, inst->crop.height); + + mbpf = NUM_MBS_PER_FRAME(height, width); + fps = inst->max_rate; + mbs_per_second = mbpf * fps; + + fw_cycles = fps * inst->cap[MB_CYCLES_FW].value; + fw_vpp_cycles = fps * inst->cap[MB_CYCLES_FW_VPP].value; + + vpp_cycles = mbs_per_second * inst->cap[MB_CYCLES_VPP].value / + inst->cap[PIPE].value; + vpp_cycles += max(vpp_cycles / 20, fw_vpp_cycles); + + if (inst->cap[PIPE].value > 1) + vpp_cycles += div_u64(vpp_cycles * 59, 1000); + + base_cycles = inst->cap[MB_CYCLES_VSP].value; + bitrate = fps * data_size * 8; + vsp_cycles = bitrate; + + if (inst->codec == VP9) { + vsp_cycles = div_u64(vsp_cycles * 170, 100); + } else { + base_cycles = 0; + vsp_cycles = div_u64(vsp_cycles, 2); + } + vsp_cycles = div_u64(vsp_cycles * 21, 20); + + if (inst->cap[STAGE].value == STAGE_1) + vsp_cycles = vsp_cycles * 3; + + vsp_cycles += mbs_per_second * base_cycles; + + freq = max3(vpp_cycles, vsp_cycles, fw_cycles); + + return freq; +} + +int iris_calc_bw_iris3(struct iris_inst *inst, struct bus_vote_data *data) +{ + const struct bw_info *bw_tbl = NULL; + unsigned int num_rows = 0; + unsigned int i, mbs, mbps; + struct iris_core *core; + + if (!data) + return 0; + + core = inst->core; + + mbs = (ALIGN(data->height, 16) / 16) * (ALIGN(data->width, 16) / 16); + mbps = mbs * data->fps; + if (mbps == 0) + return 0; + + bw_tbl = core->platform_data->bw_tbl_dec; + num_rows = core->platform_data->bw_tbl_dec_size; + + if (!bw_tbl || num_rows == 0) + return 0; + + for (i = 0; i < num_rows; i++) { + if (i != 0 && mbps > bw_tbl[i].mbs_per_sec) + break; + + if (is_10bit_colorformat(data->color_formats[0])) + data->bus_bw = bw_tbl[i].bw_ddr_10bit; + else + data->bus_bw = bw_tbl[i].bw_ddr; + } + + dev_info(core->dev, "bus_bw %llu\n", data->bus_bw); + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h new file mode 100644 index 0000000..2a37a9e --- /dev/null +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2023 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef __H_VPU_IRIS3_POWER_H__ +#define __H_VPU_IRIS3_POWER_H__ + +u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size); +int iris_calc_bw_iris3(struct iris_inst *inst, + struct bus_vote_data *vote_data); + +#endif From patchwork Mon Dec 18 11:32:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755790 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4373F374D7; Mon, 18 Dec 2023 11:40:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="XzZN8sHB" Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIAtAkP007257; Mon, 18 Dec 2023 11:40:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=iNKyHQ8h/NpFc83aGl2oTQT1CzgFzngR8xdKmuX4b8k=; b=Xz ZN8sHB0XNgIBi+6EaRknJe5/ifxN/X18P7MLOR9wqWuvk+VHC+rdpEIO54oDc6K5 s+vyKYjiJuzIHYPRurOG0b2QnA6tZzkMLCqj8D1UUnpTh8UOEgCmPjzK0QzT5JLl x/NqJ3GtzTjmDyfmk+7iGcA4vUFa4wFchllFswl6lguhKiRVwCQZCPYqUsnFaW7t 07BUa5QuNuMIcYvrw82zL0dOpmvV6eydmB599ed54WXSDqhHq9jsJQiq4oVPOssJ JxQTRGrViXMYBBL9KfzsDzTgC7jhGfGkzcITRImwNmGthXUXgCD9y6D3hxGWSQDX nFEi6Xx/unmTCH3eSKQg== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mb9g4y3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:04 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBe0FQ004875; Mon, 18 Dec 2023 11:40:01 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ac-3; Mon, 18 Dec 2023 11:40:01 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBa9rY000378; Mon, 18 Dec 2023 11:36:09 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX8rd030068; Mon, 18 Dec 2023 11:36:09 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 698372389; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 30/34] media: iris: register video encoder device to platform driver Date: Mon, 18 Dec 2023 17:02:25 +0530 Message-Id: <1702899149-21321-31-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: BM17vDS2em8D_fMU0u92xI97O_1Ahvlb X-Proofpoint-GUID: BM17vDS2em8D_fMU0u92xI97O_1Ahvlb X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 impostorscore=0 adultscore=0 mlxscore=0 clxscore=1015 spamscore=0 mlxlogscore=999 phishscore=0 lowpriorityscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Register/unregister video Encoder device. Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/iris_common.h | 5 +++ .../media/platform/qcom/vcodec/iris/iris_core.h | 2 ++ .../media/platform/qcom/vcodec/iris/iris_probe.c | 42 ++++++++++++++++------ 3 files changed, 38 insertions(+), 11 deletions(-) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h index 3ab4767..b1273d0 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -30,6 +30,11 @@ #define INPUT_TIMER_LIST_SIZE 30 +enum domain_type { + ENCODER = BIT(0), + DECODER = BIT(1), +}; + enum codec_type { H264 = BIT(0), HEVC = BIT(1), diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_core.h b/drivers/media/platform/qcom/vcodec/iris/iris_core.h index 50553d2..c56eb24 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_core.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_core.h @@ -23,6 +23,7 @@ * @irq: iris irq * @v4l2_dev: a holder for v4l2 device structure * @vdev_dec: iris video device structure for decoder + * @vdev_enc: iris video device structure for encoder * @v4l2_file_ops: iris v4l2 file ops * @v4l2_ioctl_ops: iris v4l2 ioctl ops * @bus_tbl: table of iris buses @@ -66,6 +67,7 @@ struct iris_core { int irq; struct v4l2_device v4l2_dev; struct video_device *vdev_dec; + struct video_device *vdev_enc; const struct v4l2_file_operations *v4l2_file_ops; const struct v4l2_ioctl_ops *v4l2_ioctl_ops; struct bus_info *bus_tbl; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c index 2571e27..b487e83 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_probe.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_probe.c @@ -31,16 +31,15 @@ static int init_iris_isr(struct iris_core *core) return ret; } -static int iris_register_video_device(struct iris_core *core) +static int iris_register_video_device(struct iris_core *core, enum domain_type type) { struct video_device *vdev; - int ret; + int ret = 0; vdev = video_device_alloc(); if (!vdev) return -ENOMEM; - strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name)); vdev->release = video_device_release; vdev->fops = core->v4l2_file_ops; vdev->ioctl_ops = core->v4l2_ioctl_ops; @@ -48,11 +47,24 @@ static int iris_register_video_device(struct iris_core *core) vdev->v4l2_dev = &core->v4l2_dev; vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING; - ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); - if (ret) - goto err_vdev_release; + if (type == DECODER) { + strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name)); + + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); + if (ret) + goto err_vdev_release; + + core->vdev_dec = vdev; + } else if (type == ENCODER) { + strscpy(vdev->name, "qcom-iris-encoder", sizeof(vdev->name)); + + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); + if (ret) + goto err_vdev_release; + + core->vdev_enc = vdev; + } - core->vdev_dec = vdev; video_set_drvdata(vdev, core); return ret; @@ -80,6 +92,8 @@ static void iris_remove(struct platform_device *pdev) video_unregister_device(core->vdev_dec); + video_unregister_device(core->vdev_enc); + v4l2_device_unregister(&core->v4l2_dev); iris_pm_put(core, false); @@ -185,17 +199,21 @@ static int iris_probe(struct platform_device *pdev) if (ret) goto err_runtime_disable; - ret = iris_register_video_device(core); + ret = iris_register_video_device(core, DECODER); if (ret) goto err_v4l2_unreg; + ret = iris_register_video_device(core, ENCODER); + if (ret) + goto err_vdev_unreg_dec; + platform_set_drvdata(pdev, core); dma_mask = core->cap[DMA_MASK].value; ret = dma_set_mask_and_coherent(dev, dma_mask); if (ret) - goto err_vdev_unreg; + goto err_vdev_unreg_enc; dma_set_max_seg_size(&pdev->dev, (unsigned int)DMA_BIT_MASK(32)); dma_set_seg_boundary(&pdev->dev, (unsigned long)DMA_BIT_MASK(64)); @@ -206,7 +224,7 @@ static int iris_probe(struct platform_device *pdev) if (ret) { dev_err_probe(core->dev, ret, "%s: interface queues init failed\n", __func__); - goto err_vdev_unreg; + goto err_vdev_unreg_enc; } ret = iris_pm_get(core); @@ -236,7 +254,9 @@ static int iris_probe(struct platform_device *pdev) hfi_queue_deinit(core->dev, &core->iface_q_table, &core->sfr, &core->command_queue, &core->message_queue, &core->debug_queue); -err_vdev_unreg: +err_vdev_unreg_enc: + video_unregister_device(core->vdev_enc); +err_vdev_unreg_dec: video_unregister_device(core->vdev_dec); err_v4l2_unreg: v4l2_device_unregister(&core->v4l2_dev); From patchwork Mon Dec 18 11:32:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal X-Patchwork-Id: 755786 Received: from mx0a-0031df01.pphosted.com (mx0a-0031df01.pphosted.com [205.220.168.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF2E9374D7; Mon, 18 Dec 2023 11:40:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=quicinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=qualcomm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=quicinc.com header.i=@quicinc.com header.b="N+0Eut/L" Received: from pps.filterd (m0279864.ppops.net [127.0.0.1]) by mx0a-0031df01.pphosted.com (8.17.1.24/8.17.1.24) with ESMTP id 3BIArvvt003529; Mon, 18 Dec 2023 11:40:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=quicinc.com; h= from:to:cc:subject:date:message-id:in-reply-to:references; s= qcppdkim1; bh=vLyhCEyyM++iSdmMnhPhEeX/SePVJRxrjXz9jo/+Aj8=; b=N+ 0Eut/LJZRxg02e4C1Pq0XZJWmA2o4pKqGbF+DGTm1hKe9dDm1I+C875LyLUgNZB4 i8u44/b2WaAbAmNEdlDHI7GbPhRL6YNcQq+tJ8TJmhTYG4FHg5jkFKuNdvpbEBAe wElL0Yq6p3QhGVNRoE9liJ0OMbO9WzDOUGt8RcJ2cwQeAAZZSE02SRwvtrw8hGGt zYEg+sCMeyTO+zlCCW1IwGXfjexNkArDAseOameOEc8qvVGpoNbLHyv2io+0kWFU 0ZXC572Z7yYF+I5vSMhjT/4rJGEo546GUg+ctAwD/1quxwM2T7pjPI4zLwUBI7gL //9mXB1vkyhrXtG1+xXQ== Received: from apblrppmta01.qualcomm.com (blr-bdr-fw-01_GlobalNAT_AllZones-Outside.qualcomm.com [103.229.18.19]) by mx0a-0031df01.pphosted.com (PPS) with ESMTPS id 3v2mb9g4xw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 18 Dec 2023 11:40:03 +0000 (GMT) Received: from pps.filterd (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX6PL029956; Mon, 18 Dec 2023 11:40:00 GMT Received: from pps.reinject (localhost [127.0.0.1]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3v14ym00ad-1; Mon, 18 Dec 2023 11:40:00 +0000 Received: from APBLRPPMTA01.qualcomm.com (APBLRPPMTA01.qualcomm.com [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 3BIBX8rc030072; Mon, 18 Dec 2023 11:36:09 GMT Received: from hu-sgudaval-hyd.qualcomm.com (hu-dikshita-hyd.qualcomm.com [10.213.110.13]) by APBLRPPMTA01.qualcomm.com (PPS) with ESMTP id 3BIBX853030069; Mon, 18 Dec 2023 11:36:09 +0000 Received: by hu-sgudaval-hyd.qualcomm.com (Postfix, from userid 347544) id 6C0E7238B; Mon, 18 Dec 2023 17:03:04 +0530 (+0530) From: Dikshita Agarwal To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, stanimir.k.varbanov@gmail.com, quic_vgarodia@quicinc.com, agross@kernel.org, andersson@kernel.org, konrad.dybcio@linaro.org, mchehab@kernel.org, bryan.odonoghue@linaro.org Cc: linux-arm-msm@vger.kernel.org, quic_abhinavk@quicinc.com, Dikshita Agarwal Subject: [PATCH v2 31/34] media: iris: add platform specific instance capabilities for encoder Date: Mon, 18 Dec 2023 17:02:26 +0530 Message-Id: <1702899149-21321-32-git-send-email-quic_dikshita@quicinc.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> References: <1702899149-21321-1-git-send-email-quic_dikshita@quicinc.com> X-QCInternal: smtphost X-QCInternal: smtphost X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-Virus-Version: vendor=nai engine=6200 definitions=5800 signatures=585085 X-Proofpoint-ORIG-GUID: jEWxbDd7IENax4tQu8kZHHd5C7UvPkHF X-Proofpoint-GUID: jEWxbDd7IENax4tQu8kZHHd5C7UvPkHF X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2023-12-09_01,2023-12-07_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 impostorscore=0 adultscore=0 mlxscore=0 clxscore=1015 spamscore=0 mlxlogscore=999 phishscore=0 lowpriorityscore=0 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2311290000 definitions=main-2312180084 Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Capabilities are set of video specifications and features supported by a specific platform(SOC). Each capability is defined with min, max, range, default value and corresponding HFI. Also, platform data defines different resource details for a specific platform(SOC). This change defines resource tables for sm8550 platform data and use for initializing these resources. Add Children, Set, Adjust functions for to each capability for Encoder. Signed-off-by: Dikshita Agarwal --- .../media/platform/qcom/vcodec/iris/hfi_defines.h | 128 ++- .../media/platform/qcom/vcodec/iris/iris_common.h | 4 + .../media/platform/qcom/vcodec/iris/iris_ctrls.c | 981 ++++++++++++++++++++- .../media/platform/qcom/vcodec/iris/iris_ctrls.h | 35 + .../media/platform/qcom/vcodec/iris/iris_helpers.c | 16 +- drivers/media/platform/qcom/vcodec/iris/iris_hfi.c | 53 ++ drivers/media/platform/qcom/vcodec/iris/iris_hfi.h | 7 +- .../platform/qcom/vcodec/iris/iris_hfi_packet.c | 10 +- .../platform/qcom/vcodec/iris/iris_hfi_packet.h | 5 + .../platform/qcom/vcodec/iris/iris_hfi_response.c | 6 + .../platform/qcom/vcodec/iris/iris_instance.h | 6 + .../platform/qcom/vcodec/iris/platform_common.h | 74 +- .../platform/qcom/vcodec/iris/platform_sm8550.c | 768 +++++++++++++++- .../platform/qcom/vcodec/iris/vpu_iris3_power.c | 3 + 14 files changed, 2024 insertions(+), 72 deletions(-) diff --git a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h index 872674e..bc32c99 100644 --- a/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h +++ b/drivers/media/platform/qcom/vcodec/iris/hfi_defines.h @@ -45,6 +45,8 @@ enum hfi_property_mode_type { #define HFI_BITMASK_BITSTREAM_WIDTH 0xffff0000 #define HFI_BITMASK_BITSTREAM_HEIGHT 0x0000ffff +#define HFI_LEVEL_NONE 0xFFFFFFFF + #define HFI_BITMASK_FRAME_MBS_ONLY_FLAG 0x00000001 #define HFI_TRUE 0x00000001 @@ -68,14 +70,14 @@ enum hfi_codec_type { #define HFI_PROP_CODEC 0x03000100 enum hfi_color_format { - HFI_COLOR_FMT_OPAQUE = 0, - HFI_COLOR_FMT_NV12 = 1, - HFI_COLOR_FMT_NV12_UBWC = 2, - HFI_COLOR_FMT_P010 = 3, - HFI_COLOR_FMT_TP10_UBWC = 4, - HFI_COLOR_FMT_RGBA8888 = 5, - HFI_COLOR_FMT_RGBA8888_UBWC = 6, - HFI_COLOR_FMT_NV21 = 7, + HFI_COLOR_FMT_OPAQUE = 0, + HFI_COLOR_FMT_NV12 = 1, + HFI_COLOR_FMT_NV12_UBWC = 2, + HFI_COLOR_FMT_P010 = 3, + HFI_COLOR_FMT_TP10_UBWC = 4, + HFI_COLOR_FMT_RGBA8888 = 5, + HFI_COLOR_FMT_RGBA8888_UBWC = 6, + HFI_COLOR_FMT_NV21 = 7, }; #define HFI_PROP_COLOR_FORMAT 0x03000101 @@ -104,14 +106,108 @@ enum hfi_color_format { #define HFI_PROP_CABAC_SESSION 0x03000121 +#define HFI_PROP_8X8_TRANSFORM 0x03000122 + #define HFI_PROP_BUFFER_HOST_MAX_COUNT 0x03000123 #define HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT 0x03000124 #define HFI_PROP_PIC_ORDER_CNT_TYPE 0x03000128 +enum hfi_syncframe_request_mode { + HFI_SYNC_FRAME_REQUEST_WITHOUT_SEQ_HDR = 0x00000001, + HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR = 0x00000002, +}; + +enum hfi_rate_control { + HFI_RC_VBR_CFR = 0x00000000, + HFI_RC_CBR_CFR = 0x00000001, + HFI_RC_CQ = 0x00000002, + HFI_RC_OFF = 0x00000003, + HFI_RC_CBR_VFR = 0x00000004, + HFI_RC_LOSSLESS = 0x00000005, +}; + +#define HFI_PROP_RATE_CONTROL 0x0300012a + +#define HFI_PROP_QP_PACKED 0x0300012e + +#define HFI_PROP_MIN_QP_PACKED 0x0300012f + +#define HFI_PROP_MAX_QP_PACKED 0x03000130 + +enum hfi_layer_enc_type { + HFI_HIER_P_SLIDING_WINDOW = 0x1, + HFI_HIER_P_HYBRID_LTR = 0x2, + HFI_HIER_B = 0x3, +}; + +#define HFI_PROP_IR_RANDOM_PERIOD 0x03000131 + +#define HFI_PROP_MULTI_SLICE_MB_COUNT 0x03000132 + +#define HFI_PROP_MULTI_SLICE_BYTES_COUNT 0x03000133 + +#define HFI_PROP_LTR_COUNT 0x03000134 + +#define HFI_PROP_LTR_MARK 0x03000135 + +#define HFI_PROP_LTR_USE 0x03000136 + +#define HFI_PROP_LAYER_ENCODING_TYPE 0x03000138 + +#define HFI_PROP_LAYER_COUNT 0x03000139 + +#define HFI_PROP_TOTAL_BITRATE 0x0300013b + +#define HFI_PROP_BITRATE_LAYER1 0x0300013c + +#define HFI_PROP_BITRATE_LAYER2 0x0300013d + +#define HFI_PROP_BITRATE_LAYER3 0x0300013e + +#define HFI_PROP_BITRATE_LAYER4 0x0300013f + +#define HFI_PROP_BITRATE_LAYER5 0x03000140 + +#define HFI_PROP_BITRATE_LAYER6 0x03000141 + +#define HFI_PROP_BASELAYER_PRIORITYID 0x03000142 + +#define HFI_PROP_REQUEST_SYNC_FRAME 0x03000145 + +#define HFI_PROP_MAX_GOP_FRAMES 0x03000146 + +#define HFI_PROP_MAX_B_FRAMES 0x03000147 + #define HFI_PROP_QUALITY_MODE 0x03000148 +enum hfi_seq_header_mode { + HFI_SEQ_HEADER_SEPERATE_FRAME = 0x00000001, + HFI_SEQ_HEADER_JOINED_WITH_1ST_FRAME = 0x00000002, + HFI_SEQ_HEADER_PREFIX_WITH_SYNC_FRAME = 0x00000004, + HFI_SEQ_HEADER_METADATA = 0x00000008, +}; + +#define HFI_PROP_SEQ_HEADER_MODE 0x03000149 + +enum hfi_rotation { + HFI_ROTATION_NONE = 0x00000000, + HFI_ROTATION_90 = 0x00000001, + HFI_ROTATION_180 = 0x00000002, + HFI_ROTATION_270 = 0x00000003, +}; + +#define HFI_PROP_ROTATION 0x0300014b + +enum hfi_flip { + HFI_DISABLE_FLIP = 0x00000000, + HFI_HORIZONTAL_FLIP = 0x00000001, + HFI_VERTICAL_FLIP = 0x00000002, +}; + +#define HFI_PROP_FLIP 0x0300014c + enum hfi_color_primaries { HFI_PRIMARIES_RESERVED = 0, HFI_PRIMARIES_BT709 = 1, @@ -182,7 +278,7 @@ enum hfi_picture_type { #define HFI_PROP_PICTURE_TYPE 0x03000162 -#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168 +#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168 #define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169 @@ -190,6 +286,12 @@ enum hfi_picture_type { #define HFI_PROP_DPB_LIST 0x0300017A +#define HFI_PROP_TOTAL_PEAK_BITRATE 0x0300017C + +#define HFI_PROP_MAINTAIN_MIN_QUALITY 0x0300017D + +#define HFI_PROP_IR_CYCLIC_PERIOD 0x0300017E + #define HFI_PROP_UBWC_STRIDE_SCANLINE 0x03000190 #define HFI_PROP_COMV_BUFFER_COUNT 0x03000193 @@ -214,7 +316,7 @@ enum hfi_picture_type { #define HFI_SYSTEM_ERROR_BEGIN 0x05000000 -#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001 +#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001 #define HFI_SYSTEM_ERROR_END 0x05FFFFFF @@ -233,9 +335,9 @@ enum hfi_picture_type { #define HFI_INFORMATION_END 0x06FFFFFF struct hfi_debug_header { - u32 size; - u32 debug_level; - u32 reserved[2]; + u32 size; + u32 debug_level; + u32 reserved[2]; }; enum hfi_buffer_type { diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_common.h b/drivers/media/platform/qcom/vcodec/iris/iris_common.h index b1273d0..ca5406a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_common.h @@ -30,6 +30,10 @@ #define INPUT_TIMER_LIST_SIZE 30 +#define CABAC_MAX_BITRATE 160000000 + +#define CAVLC_MAX_BITRATE 220000000 + enum domain_type { ENCODER = BIT(0), DECODER = BIT(1), diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c index 94fff74..a648cc1 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.c @@ -611,6 +611,45 @@ int prepare_dependency_list(struct iris_inst *inst) return ret; } +static inline bool is_layer_bitrate_set(struct iris_inst *inst) +{ + u32 layer_br_caps[6] = {L0_BR, L1_BR, L2_BR, L3_BR, L4_BR, L5_BR}; + u32 cap_id = 0, i, enh_layer_count; + + enh_layer_count = inst->cap[ENH_LAYER_COUNT].value; + + for (i = 0; i <= enh_layer_count; i++) { + if (i >= ARRAY_SIZE(layer_br_caps)) + break; + + cap_id = layer_br_caps[i]; + if (!(inst->cap[cap_id].flags & CAP_FLAG_CLIENT_SET)) + return false; + } + + return true; +} + +static inline u32 get_cumulative_bitrate(struct iris_inst *inst) +{ + u32 layer_br_caps[6] = {L0_BR, L1_BR, L2_BR, L3_BR, L4_BR, L5_BR}; + u32 cumulative_br = 0; + s32 enh_layer_count; + u32 cap_id = 0; + int i; + + enh_layer_count = inst->cap[ENH_LAYER_COUNT].value; + + for (i = 0; i <= enh_layer_count; i++) { + if (i >= ARRAY_SIZE(layer_br_caps)) + break; + cap_id = layer_br_caps[i]; + cumulative_br += inst->cap[cap_id].value; + } + + return cumulative_br; +} + int set_u32_enum(struct iris_inst *inst, enum plat_inst_cap_type cap_id) { @@ -635,6 +674,18 @@ int set_u32(struct iris_inst *inst, &hfi_value, sizeof(u32)); } +int set_q16(struct iris_inst *inst, + enum plat_inst_cap_type cap_id) +{ + u32 hfi_value = inst->cap[cap_id].value; + u32 hfi_id = inst->cap[cap_id].hfi_id; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_Q16, + &hfi_value, sizeof(u32)); +} + int set_stage(struct iris_inst *inst, enum plat_inst_cap_type cap_id) { @@ -662,7 +713,7 @@ int set_pipe(struct iris_inst *inst, { u32 work_route, hfi_id; - work_route = inst->cap[PIPE].value; + work_route = inst->cap[cap_id].value; hfi_id = inst->cap[cap_id].hfi_id; return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, @@ -671,6 +722,419 @@ int set_pipe(struct iris_inst *inst, &work_route, sizeof(u32)); } +int set_level(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_value = inst->cap[cap_id].value; + u32 hfi_id = inst->cap[cap_id].hfi_id; + + if (!(inst->cap[cap_id].flags & CAP_FLAG_CLIENT_SET)) + hfi_value = HFI_LEVEL_NONE; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32_ENUM, + &hfi_value, sizeof(u32)); +} + +int set_req_sync_frame(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_id, hfi_val; + s32 prepend_sps_pps; + + prepend_sps_pps = inst->cap[PREPEND_SPSPPS_TO_IDR].value; + hfi_id = inst->cap[cap_id].hfi_id; + + if (prepend_sps_pps) + hfi_val = HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR; + else + hfi_val = HFI_SYNC_FRAME_REQUEST_WITHOUT_SEQ_HDR; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32_ENUM, + &hfi_val, sizeof(u32)); +} + +int set_flip(struct iris_inst *inst, + enum plat_inst_cap_type cap_id) +{ + u32 hflip, vflip, ret = 0; + + u32 hfi_value = HFI_DISABLE_FLIP; + u32 hfi_id = inst->cap[cap_id].hfi_id; + + hflip = inst->cap[HFLIP].value; + vflip = inst->cap[VFLIP].value; + + if (hflip) + hfi_value |= HFI_HORIZONTAL_FLIP; + + if (vflip) + hfi_value |= HFI_VERTICAL_FLIP; + + if (inst->vb2q_dst->streaming) { + if (hfi_value != HFI_DISABLE_FLIP) { + ret = set_req_sync_frame(inst, REQUEST_I_FRAME); + if (ret) + return ret; + } + } + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32_ENUM, + &hfi_value, sizeof(u32)); +} + +int set_rotation(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_id, hfi_val; + + hfi_id = inst->cap[cap_id].hfi_id; + + switch (inst->cap[cap_id].value) { + case 0: + hfi_val = HFI_ROTATION_NONE; + break; + case 90: + hfi_val = HFI_ROTATION_90; + break; + case 180: + hfi_val = HFI_ROTATION_180; + break; + case 270: + hfi_val = HFI_ROTATION_270; + break; + default: + hfi_val = HFI_ROTATION_NONE; + break; + } + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &hfi_val, sizeof(u32)); +} + +int set_header_mode(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 header_mode, hfi_id, hfi_val; + s32 prepend_sps_pps; + + prepend_sps_pps = inst->cap[PREPEND_SPSPPS_TO_IDR].value; + header_mode = inst->cap[cap_id].value; + hfi_id = inst->cap[cap_id].hfi_id; + + if (prepend_sps_pps) + hfi_val = HFI_SEQ_HEADER_PREFIX_WITH_SYNC_FRAME; + else if (header_mode == V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME) + hfi_val = HFI_SEQ_HEADER_JOINED_WITH_1ST_FRAME; + else + hfi_val = HFI_SEQ_HEADER_SEPERATE_FRAME; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32_ENUM, + &hfi_val, sizeof(u32)); +} + +int set_gop_size(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_value, hfi_id; + + if (inst->vb2q_dst->streaming) { + if (inst->hfi_layer_type == HFI_HIER_B) + return 0; + } + + hfi_value = inst->cap[GOP_SIZE].value; + hfi_id = inst->cap[cap_id].hfi_id; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &hfi_value, sizeof(u32)); +} + +int set_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_id, hfi_val; + + if (inst->cap[BIT_RATE].flags & CAP_FLAG_CLIENT_SET) + goto set_total_bitrate; + + if (inst->vb2q_dst->streaming) + return 0; + +set_total_bitrate: + hfi_id = inst->cap[cap_id].hfi_id; + hfi_val = inst->cap[cap_id].value; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &hfi_val, sizeof(u32)); +} + +int set_layer_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_value = 0; + u32 hfi_id; + + if (!inst->vb2q_dst->streaming) + return 0; + + if (inst->cap[BIT_RATE].flags & CAP_FLAG_CLIENT_SET) + return 0; + + if (!inst->cap[ENH_LAYER_COUNT].max || + !is_layer_bitrate_set(inst)) + return 0; + + hfi_value = inst->cap[BIT_RATE].value; + hfi_id = inst->cap[BIT_RATE].hfi_id; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &hfi_value, sizeof(u32)); +} + +int set_peak_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_id, hfi_val; + s32 rc_mode; + + hfi_id = inst->cap[cap_id].hfi_id; + hfi_val = inst->cap[cap_id].value; + + rc_mode = inst->cap[BITRATE_MODE].value; + if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_CBR) + return 0; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &hfi_val, sizeof(u32)); +} + +int set_use_and_mark_ltr(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_id, hfi_val; + + hfi_id = inst->cap[cap_id].hfi_id; + hfi_val = inst->cap[cap_id].value; + + if (!inst->cap[LTR_COUNT].value || + inst->cap[cap_id].value == INVALID_DEFAULT_MARK_OR_USE_LTR) + return 0; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &hfi_val, sizeof(u32)); +} + +int set_ir_period(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_id = 0, hfi_val; + + hfi_val = inst->cap[cap_id].value; + + if (inst->cap[IR_TYPE].value == + V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM) { + hfi_id = HFI_PROP_IR_RANDOM_PERIOD; + } else if (inst->cap[IR_TYPE].value == + V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC) { + hfi_id = HFI_PROP_IR_CYCLIC_PERIOD; + } + + return iris_hfi_set_ir_period(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &hfi_val, sizeof(u32)); +} + +int set_min_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + s32 i_frame_qp = 0, p_frame_qp = 0, b_frame_qp = 0, min_qp_enable = 0; + u32 i_qp_enable = 0, p_qp_enable = 0, b_qp_enable = 0; + u32 client_qp_enable = 0, hfi_value = 0; + u32 hfi_id; + + if (inst->cap[MIN_FRAME_QP].flags & CAP_FLAG_CLIENT_SET) + min_qp_enable = 1; + + if (min_qp_enable || + (inst->cap[I_FRAME_MIN_QP].flags & CAP_FLAG_CLIENT_SET)) + i_qp_enable = 1; + if (min_qp_enable || + (inst->cap[P_FRAME_MIN_QP].flags & CAP_FLAG_CLIENT_SET)) + p_qp_enable = 1; + if (min_qp_enable || + (inst->cap[B_FRAME_MIN_QP].flags & CAP_FLAG_CLIENT_SET)) + b_qp_enable = 1; + + client_qp_enable = i_qp_enable | p_qp_enable << 1 | b_qp_enable << 2; + if (!client_qp_enable) + return 0; + + i_frame_qp = max(inst->cap[I_FRAME_MIN_QP].value, inst->cap[MIN_FRAME_QP].value); + p_frame_qp = max(inst->cap[P_FRAME_MIN_QP].value, inst->cap[MIN_FRAME_QP].value); + b_frame_qp = max(inst->cap[B_FRAME_MIN_QP].value, inst->cap[MIN_FRAME_QP].value); + + hfi_id = inst->cap[cap_id].hfi_id; + hfi_value = i_frame_qp | p_frame_qp << 8 | b_frame_qp << 16 | + client_qp_enable << 24; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_32_PACKED, + &hfi_value, sizeof(u32)); +} + +int set_max_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + s32 i_frame_qp = 0, p_frame_qp = 0, b_frame_qp = 0, max_qp_enable = 0; + u32 i_qp_enable = 0, p_qp_enable = 0, b_qp_enable = 0; + u32 client_qp_enable = 0, hfi_value = 0; + u32 hfi_id; + + if (inst->cap[MAX_FRAME_QP].flags & CAP_FLAG_CLIENT_SET) + max_qp_enable = 1; + + if (max_qp_enable || + (inst->cap[I_FRAME_MAX_QP].flags & CAP_FLAG_CLIENT_SET)) + i_qp_enable = 1; + if (max_qp_enable || + (inst->cap[P_FRAME_MAX_QP].flags & CAP_FLAG_CLIENT_SET)) + p_qp_enable = 1; + if (max_qp_enable || + (inst->cap[B_FRAME_MAX_QP].flags & CAP_FLAG_CLIENT_SET)) + b_qp_enable = 1; + + client_qp_enable = i_qp_enable | p_qp_enable << 1 | b_qp_enable << 2; + if (!client_qp_enable) + return 0; + + i_frame_qp = min(inst->cap[I_FRAME_MAX_QP].value, inst->cap[MAX_FRAME_QP].value); + p_frame_qp = min(inst->cap[P_FRAME_MAX_QP].value, inst->cap[MAX_FRAME_QP].value); + b_frame_qp = min(inst->cap[B_FRAME_MAX_QP].value, inst->cap[MAX_FRAME_QP].value); + + hfi_id = inst->cap[cap_id].hfi_id; + hfi_value = i_frame_qp | p_frame_qp << 8 | b_frame_qp << 16 | + client_qp_enable << 24; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_32_PACKED, + &hfi_value, sizeof(u32)); +} + +int set_frame_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 i_qp_enable = 0, p_qp_enable = 0, b_qp_enable = 0; + u32 client_qp_enable = 0, hfi_value = 0; + s32 i_frame_qp = 0, p_frame_qp = 0, b_frame_qp = 0; + s32 rc_type = -1; + u32 hfi_id; + + rc_type = inst->hfi_rc_type; + if (inst->vb2q_dst->streaming) { + if (rc_type != HFI_RC_OFF) + return 0; + } + + if (rc_type == HFI_RC_OFF) { + i_qp_enable = 1; + p_qp_enable = 1; + b_qp_enable = 1; + } else { + if (inst->cap[I_FRAME_QP].flags & CAP_FLAG_CLIENT_SET) + i_qp_enable = 1; + if (inst->cap[P_FRAME_QP].flags & CAP_FLAG_CLIENT_SET) + p_qp_enable = 1; + if (inst->cap[B_FRAME_QP].flags & CAP_FLAG_CLIENT_SET) + b_qp_enable = 1; + } + + client_qp_enable = i_qp_enable | p_qp_enable << 1 | b_qp_enable << 2; + if (!client_qp_enable) + return 0; + + i_frame_qp = inst->cap[I_FRAME_QP].value; + p_frame_qp = inst->cap[P_FRAME_QP].value; + b_frame_qp = inst->cap[B_FRAME_QP].value; + + hfi_id = inst->cap[cap_id].hfi_id; + hfi_value = i_frame_qp | p_frame_qp << 8 | b_frame_qp << 16 | + client_qp_enable << 24; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, cap_id), + HFI_PAYLOAD_32_PACKED, + &hfi_value, sizeof(u32)); +} + +int set_layer_count_and_type(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_layer_count, hfi_layer_type = 0; + int ret, hfi_id; + + if (!inst->vb2q_dst->streaming) { + hfi_layer_type = inst->hfi_layer_type; + hfi_id = inst->cap[LAYER_TYPE].hfi_id; + + ret = iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, LAYER_TYPE), + HFI_PAYLOAD_U32_ENUM, + &hfi_layer_type, sizeof(u32)); + if (ret) + return ret; + } else { + if (inst->hfi_layer_type == HFI_HIER_B) + return 0; + } + + hfi_id = inst->cap[ENH_LAYER_COUNT].hfi_id; + hfi_layer_count = inst->cap[ENH_LAYER_COUNT].value + 1; + + ret = iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, ENH_LAYER_COUNT), + HFI_PAYLOAD_U32, + &hfi_layer_count, sizeof(u32)); + + return ret; +} + +int set_slice_count(struct iris_inst *inst, enum plat_inst_cap_type cap_id) +{ + u32 hfi_value = 0, set_cap_id = 0, hfi_id; + s32 slice_mode = -1; + + slice_mode = inst->cap[SLICE_MODE].value; + + if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE) + return 0; + + if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) { + hfi_value = (inst->codec == HEVC) ? + (inst->cap[SLICE_MAX_MB].value + 3) / 4 : + inst->cap[SLICE_MAX_MB].value; + set_cap_id = SLICE_MAX_MB; + } else if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES) { + hfi_value = inst->cap[SLICE_MAX_BYTES].value; + set_cap_id = SLICE_MAX_BYTES; + } + + hfi_id = inst->cap[set_cap_id].hfi_id; + + return iris_hfi_set_property(inst, hfi_id, HFI_HOST_FLAGS_NONE, + get_port_info(inst, set_cap_id), + HFI_PAYLOAD_U32, + &hfi_value, sizeof(u32)); +} + int set_v4l2_properties(struct iris_inst *inst) { struct cap_entry *entry = NULL, *temp = NULL; @@ -723,8 +1187,7 @@ int adjust_profile(struct iris_inst *inst, struct v4l2_ctrl *ctrl) u32 adjusted_value; s32 pix_fmt = -1; - adjusted_value = ctrl ? ctrl->val : - inst->cap[PROFILE].value; + adjusted_value = ctrl ? ctrl->val : inst->cap[PROFILE].value; pix_fmt = inst->cap[PIX_FMTS].value; @@ -737,3 +1200,515 @@ int adjust_profile(struct iris_inst *inst, struct v4l2_ctrl *ctrl) return 0; } + +int adjust_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + u32 layer_br_caps[6] = {L0_BR, L1_BR, L2_BR, L3_BR, L4_BR, L5_BR}; + u32 adjusted_value, cumulative_bitrate, cap_id, cap_val, i; + s32 layer_count, max_bitrate = 0, entropy_mode; + + adjusted_value = ctrl ? ctrl->val : inst->cap[BIT_RATE].value; + + if (inst->cap[BIT_RATE].flags & CAP_FLAG_CLIENT_SET) { + inst->cap[BIT_RATE].value = adjusted_value; + return 0; + } + + entropy_mode = inst->cap[ENTROPY_MODE].value; + + if (inst->codec == HEVC) + max_bitrate = CABAC_MAX_BITRATE; + + if (inst->codec == H264) { + if (entropy_mode == V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC) + max_bitrate = CABAC_MAX_BITRATE; + else + max_bitrate = CAVLC_MAX_BITRATE; + } + + if (inst->cap[BIT_RATE].value > max_bitrate) + inst->cap[BIT_RATE].value = max_bitrate; + + layer_count = inst->cap[ENH_LAYER_COUNT].value; + if (!layer_count) + return 0; + + if (!is_layer_bitrate_set(inst)) + return 0; + + cumulative_bitrate = get_cumulative_bitrate(inst); + + if (cumulative_bitrate > max_bitrate) { + u32 decrement_in_value = 0; + u32 decrement_in_percent = ((cumulative_bitrate - max_bitrate) * 100) / + max_bitrate; + + cumulative_bitrate = 0; + for (i = 0; i <= layer_count; i++) { + if (i >= ARRAY_SIZE(layer_br_caps)) + break; + cap_id = layer_br_caps[i]; + cap_val = inst->cap[cap_id].value; + decrement_in_value = (cap_val * decrement_in_percent) / 100; + cumulative_bitrate += (cap_val - decrement_in_value); + inst->cap[cap_id].value = cap_val - decrement_in_value; + } + inst->cap[BIT_RATE].value = cumulative_bitrate; + } + + return 0; +} + +int adjust_layer_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + u32 old_br = 0, new_br = 0, exceeded_br = 0; + u32 client_set_cap_id = INST_CAP_NONE; + u32 cumulative_bitrate = 0; + s32 max_bitrate; + + if (!ctrl) + return 0; + + if (inst->cap[BIT_RATE].flags & CAP_FLAG_CLIENT_SET || + !inst->vb2q_dst->streaming) + return 0; + + if (!inst->cap[ENH_LAYER_COUNT].max) + return -EINVAL; + + if (!is_layer_bitrate_set(inst)) + return 0; + + client_set_cap_id = get_cap_id(inst, ctrl->id); + if (!is_valid_cap_id(client_set_cap_id)) + return -EINVAL; + + cumulative_bitrate = get_cumulative_bitrate(inst); + max_bitrate = inst->cap[BIT_RATE].max; + old_br = inst->cap[client_set_cap_id].value; + new_br = ctrl->val; + + if ((cumulative_bitrate - old_br + new_br) > max_bitrate) { + exceeded_br = (cumulative_bitrate - old_br + new_br) - max_bitrate; + new_br = ctrl->val - exceeded_br; + } + inst->cap[client_set_cap_id].value = new_br; + + inst->cap[BIT_RATE].value = get_cumulative_bitrate(inst); + + return 0; +} + +int adjust_peak_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + u32 adjusted_value; + s32 rc_mode, bitrate; + + adjusted_value = ctrl ? ctrl->val : inst->cap[PEAK_BITRATE].value; + + rc_mode = inst->cap[BITRATE_MODE].value; + + if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_CBR) + return 0; + + bitrate = inst->cap[BIT_RATE].value; + + if (inst->cap[PEAK_BITRATE].flags & CAP_FLAG_CLIENT_SET) { + if (adjusted_value < bitrate) + adjusted_value = bitrate; + } else { + adjusted_value = inst->cap[BIT_RATE].value; + } + + inst->cap[PEAK_BITRATE].value = adjusted_value; + + return 0; +} + +int adjust_bitrate_mode(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 frame_rc, bitrate_mode, frame_skip; + + bitrate_mode = inst->cap[BITRATE_MODE].value; + frame_rc = inst->cap[FRAME_RC_ENABLE].value; + frame_skip = inst->cap[FRAME_SKIP_MODE].value; + + if (!frame_rc) { + inst->hfi_rc_type = HFI_RC_OFF; + return 0; + } + + if (bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_VBR) { + inst->hfi_rc_type = HFI_RC_VBR_CFR; + } else if (bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CBR) { + if (frame_skip) + inst->hfi_rc_type = HFI_RC_CBR_VFR; + else + inst->hfi_rc_type = HFI_RC_CBR_CFR; + } else if (bitrate_mode == V4L2_MPEG_VIDEO_BITRATE_MODE_CQ) { + inst->hfi_rc_type = HFI_RC_CQ; + } + + return 0; +} + +int adjust_gop_size(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value, enh_layer_count; + u32 min_gop_size, num_subgops; + + adjusted_value = ctrl ? ctrl->val : inst->cap[GOP_SIZE].value; + + enh_layer_count = inst->cap[ENH_LAYER_COUNT].value; + + if (!enh_layer_count) + goto exit; + + /* + * Layer encoding needs GOP size to be multiple of subgop size + * And subgop size is 2 ^ number of enhancement layers. + */ + min_gop_size = 1 << enh_layer_count; + num_subgops = (adjusted_value + (min_gop_size >> 1)) / min_gop_size; + if (num_subgops) + adjusted_value = num_subgops * min_gop_size; + else + adjusted_value = min_gop_size; + +exit: + inst->cap[GOP_SIZE].value = adjusted_value; + + return 0; +} + +int adjust_b_frame(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value, enh_layer_count = -1; + const u32 max_bframe_size = 7; + + adjusted_value = ctrl ? ctrl->val : inst->cap[B_FRAME].value; + + enh_layer_count = inst->cap[ENH_LAYER_COUNT].value; + + if (!enh_layer_count || inst->hfi_layer_type != HFI_HIER_B) { + adjusted_value = 0; + goto exit; + } + + adjusted_value = (1 << enh_layer_count) - 1; + + if (adjusted_value > max_bframe_size) + adjusted_value = max_bframe_size; + +exit: + inst->cap[B_FRAME].value = adjusted_value; + + return 0; +} + +int adjust_ltr_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value, rc_mode, pix_fmt; + + adjusted_value = ctrl ? ctrl->val : inst->cap[LTR_COUNT].value; + + rc_mode = inst->cap[BITRATE_MODE].value; + + if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_CBR || + inst->hfi_rc_type != HFI_RC_OFF) + adjusted_value = 0; + + pix_fmt = inst->cap[PIX_FMTS].value; + if (is_10bit_colorformat(pix_fmt)) + adjusted_value = 0; + + inst->cap[LTR_COUNT].value = adjusted_value; + + return 0; +} + +int adjust_use_ltr(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value, ltr_count; + + adjusted_value = ctrl ? ctrl->val : inst->cap[USE_LTR].value; + + ltr_count = inst->cap[LTR_COUNT].value; + if (!ltr_count) + return 0; + + /* + * USE_LTR is bitmask value, hence should be + * > 0 and <= (2 ^ LTR_COUNT) - 1 + */ + if (adjusted_value <= 0 || + adjusted_value > (1 << ltr_count) - 1) + return 0; + + inst->cap[USE_LTR].value = adjusted_value; + + return 0; +} + +int adjust_mark_ltr(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value, ltr_count; + + adjusted_value = ctrl ? ctrl->val : inst->cap[MARK_LTR].value; + + ltr_count = inst->cap[LTR_COUNT].value; + if (!ltr_count) + return 0; + + if (adjusted_value < 0 || adjusted_value > ltr_count - 1) + return 0; + + inst->cap[MARK_LTR].value = adjusted_value; + + return 0; +} + +int adjust_ir_period(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value, pix_fmt, rc_mode; + + adjusted_value = ctrl ? ctrl->val : inst->cap[IR_PERIOD].value; + + pix_fmt = inst->cap[PIX_FMTS].value; + if (is_10bit_colorformat(pix_fmt)) + adjusted_value = 0; + + rc_mode = inst->cap[BITRATE_MODE].value; + if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_CBR) + adjusted_value = 0; + + inst->cap[IR_PERIOD].value = adjusted_value; + + return 0; +} + +int adjust_min_quality(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value, pix_fmt, rc_mode, layer_count; + u32 width, height, frame_rate; + struct v4l2_format *f; + + if (inst->vb2q_dst->streaming) + return 0; + + adjusted_value = MAX_SUPPORTED_MIN_QUALITY; + + rc_mode = inst->cap[BITRATE_MODE].value; + if (rc_mode != V4L2_MPEG_VIDEO_BITRATE_MODE_VBR) + adjusted_value = 0; + + layer_count = inst->cap[ENH_LAYER_COUNT].value; + if (layer_count && inst->hfi_layer_type != HFI_HIER_B) + adjusted_value = 0; + + pix_fmt = inst->cap[PIX_FMTS].value; + if (is_10bit_colorformat(pix_fmt)) + adjusted_value = 0; + + frame_rate = inst->cap[FRAME_RATE].value >> 16; + f = inst->fmt_dst; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + if (!res_is_less_than(width, height, 1920, 1080)) + adjusted_value = 0; + + if (frame_rate > 60) + adjusted_value = 0; + + inst->cap[MIN_QUALITY].value = adjusted_value; + + return 0; +} + +static int adjust_static_layer_count_and_type(struct iris_inst *inst, s32 layer_count) +{ + bool hb_requested = false; + s32 max_enh_count = 0; + + if (!layer_count) + goto exit; + + if (inst->hfi_rc_type == HFI_RC_CQ) { + layer_count = 0; + goto exit; + } + + if (inst->codec == H264) { + if (!inst->cap[LAYER_ENABLE].value) { + layer_count = 0; + goto exit; + } + hb_requested = inst->cap[LAYER_TYPE].value == + V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B; + } else if (inst->codec == HEVC) { + hb_requested = inst->cap[LAYER_TYPE].value == + V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B; + } + + if (hb_requested && inst->hfi_rc_type != HFI_RC_VBR_CFR) { + layer_count = 0; + goto exit; + } + + inst->hfi_layer_type = hb_requested ? HFI_HIER_B : + (inst->codec == H264 && inst->hfi_rc_type == HFI_RC_VBR_CFR) ? + HFI_HIER_P_HYBRID_LTR : HFI_HIER_P_SLIDING_WINDOW; + + max_enh_count = inst->hfi_layer_type == HFI_HIER_B ? MAX_ENH_LAYER_HB : + inst->hfi_layer_type == HFI_HIER_P_HYBRID_LTR ? MAX_AVC_ENH_LAYER_HYBRID_HP : + inst->hfi_layer_type == HFI_HIER_P_SLIDING_WINDOW ? + (inst->codec == H264 ? MAX_AVC_ENH_LAYER_SLIDING_WINDOW : + (inst->codec == HEVC && inst->hfi_rc_type == HFI_RC_VBR_CFR) ? + MAX_HEVC_VBR_ENH_LAYER_SLIDING_WINDOW : + MAX_HEVC_NON_VBR_ENH_LAYER_SLIDING_WINDOW) : + layer_count; + + layer_count = min(layer_count, max_enh_count); + +exit: + inst->cap[ENH_LAYER_COUNT].value = layer_count; + inst->cap[ENH_LAYER_COUNT].max = layer_count; + + return 0; +} + +int adjust_layer_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 client_layer_count; + int ret = 0; + + client_layer_count = ctrl ? ctrl->val : inst->cap[ENH_LAYER_COUNT].value; + + if (!inst->vb2q_dst->streaming) { + ret = adjust_static_layer_count_and_type(inst, client_layer_count); + if (ret) + return ret; + } else { + if (inst->hfi_rc_type == HFI_RC_CBR_CFR || + inst->hfi_rc_type == HFI_RC_CBR_VFR) + return ret; + + if (inst->hfi_layer_type == HFI_HIER_P_HYBRID_LTR || + inst->hfi_layer_type == HFI_HIER_P_SLIDING_WINDOW) + inst->cap[ENH_LAYER_COUNT].value = + min(client_layer_count, inst->cap[ENH_LAYER_COUNT].max); + } + + return ret; +} + +int adjust_entropy_mode(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value; + s32 profile = -1; + + adjusted_value = ctrl ? ctrl->val : inst->cap[ENTROPY_MODE].value; + + profile = inst->cap[PROFILE].value; + if (profile == V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE || + profile == V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) + adjusted_value = V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC; + + inst->cap[ENTROPY_MODE].value = adjusted_value; + + return 0; +} + +int adjust_slice_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value, rc_type = -1, slice_mode, enh_layer_count = 0; + u32 slice_val, mbpf = 0, mbps = 0, max_mbpf = 0, max_mbps = 0, bitrate = 0; + u32 update_cap, max_avg_slicesize, output_width, output_height; + u32 min_width, min_height, max_width, max_height, fps; + + slice_mode = ctrl ? ctrl->val : inst->cap[SLICE_MODE].value; + if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE) + return 0; + + bitrate = inst->cap[BIT_RATE].value; + enh_layer_count = inst->cap[ENH_LAYER_COUNT].value; + if (enh_layer_count && is_layer_bitrate_set(inst)) + bitrate = get_cumulative_bitrate(inst); + + rc_type = inst->hfi_rc_type; + fps = inst->cap[FRAME_RATE].value >> 16; + if (fps > MAX_SLICES_FRAME_RATE || + (rc_type != HFI_RC_OFF && rc_type != HFI_RC_CBR_CFR && + rc_type != HFI_RC_CBR_VFR && rc_type != HFI_RC_VBR_CFR)) { + adjusted_value = V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE; + update_cap = SLICE_MODE; + goto exit; + } + + output_width = inst->fmt_dst->fmt.pix_mp.width; + output_height = inst->fmt_dst->fmt.pix_mp.height; + + max_width = (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) ? + MAX_MB_SLICE_WIDTH : MAX_BYTES_SLICE_WIDTH; + max_height = (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) ? + MAX_MB_SLICE_HEIGHT : MAX_BYTES_SLICE_HEIGHT; + min_width = (inst->codec == HEVC) ? + MIN_HEVC_SLICE_WIDTH : MIN_AVC_SLICE_WIDTH; + min_height = MIN_SLICE_HEIGHT; + + if (output_width < min_width || output_height < min_height || + output_width > max_width || output_height > max_width) { + adjusted_value = V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE; + update_cap = SLICE_MODE; + goto exit; + } + + mbpf = NUM_MBS_PER_FRAME(output_height, output_width); + mbps = mbpf * fps; + max_mbpf = NUM_MBS_PER_FRAME(max_height, max_width); + max_mbps = max_mbpf * MAX_SLICES_FRAME_RATE; + + if (mbpf > max_mbpf || mbps > max_mbps) { + adjusted_value = V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE; + update_cap = SLICE_MODE; + goto exit; + } + + if (slice_mode == V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) { + update_cap = SLICE_MAX_MB; + slice_val = inst->cap[SLICE_MAX_MB].value; + slice_val = max(slice_val, mbpf / MAX_SLICES_PER_FRAME); + } else { + slice_val = inst->cap[SLICE_MAX_BYTES].value; + update_cap = SLICE_MAX_BYTES; + if (rc_type != HFI_RC_OFF) { + max_avg_slicesize = + ((bitrate / fps) / 8) / MAX_SLICES_PER_FRAME; + slice_val = max(slice_val, max_avg_slicesize); + } + } + adjusted_value = slice_val; + +exit: + inst->cap[update_cap].value = adjusted_value; + + return 0; +} + +int adjust_transform_8x8(struct iris_inst *inst, struct v4l2_ctrl *ctrl) +{ + s32 adjusted_value; + s32 profile = -1; + + adjusted_value = ctrl ? ctrl->val : inst->cap[TRANSFORM_8X8].value; + + profile = inst->cap[PROFILE].value; + if (profile != V4L2_MPEG_VIDEO_H264_PROFILE_HIGH && + profile != V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH) + adjusted_value = 0; + + inst->cap[TRANSFORM_8X8].value = adjusted_value; + + return 0; +} diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h index 28ce767..5421d9f 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_ctrls.h @@ -31,5 +31,40 @@ int get_inst_capability(struct iris_inst *inst); int set_v4l2_properties(struct iris_inst *inst); int adjust_v4l2_properties(struct iris_inst *inst); int ctrls_init(struct iris_inst *inst, bool init); +int set_q16(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_level(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_req_sync_frame(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_flip(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_rotation(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_header_mode(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_gop_size(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_layer_bitrate(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_peak_bitrate(struct iris_inst *inst, + enum plat_inst_cap_type cap_id); +int set_use_and_mark_ltr(struct iris_inst *inst, + enum plat_inst_cap_type cap_id); +int set_ir_period(struct iris_inst *inst, + enum plat_inst_cap_type cap_id); +int set_min_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_max_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_frame_qp(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_layer_count_and_type(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int set_slice_count(struct iris_inst *inst, enum plat_inst_cap_type cap_id); +int adjust_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_layer_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_peak_bitrate(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_bitrate_mode(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_gop_size(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_b_frame(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_ltr_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_use_ltr(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_mark_ltr(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_ir_period(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_min_quality(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_layer_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_entropy_mode(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_slice_count(struct iris_inst *inst, struct v4l2_ctrl *ctrl); +int adjust_transform_8x8(struct iris_inst *inst, struct v4l2_ctrl *ctrl); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c index 395a189..d9d22e2 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_helpers.c @@ -40,12 +40,20 @@ bool res_is_less_than(u32 width, u32 height, u32 get_port_info(struct iris_inst *inst, enum plat_inst_cap_type cap_id) { + if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT && + inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT) { + if (inst->vb2q_dst->streaming) + return get_hfi_port(INPUT_MPLANE); + else + return get_hfi_port(OUTPUT_MPLANE); + } + if (inst->cap[cap_id].flags & CAP_FLAG_INPUT_PORT) - return HFI_PORT_BITSTREAM; + return get_hfi_port(INPUT_MPLANE); else if (inst->cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT) - return HFI_PORT_RAW; - - return HFI_PORT_NONE; + return get_hfi_port(OUTPUT_MPLANE); + else + return HFI_PORT_NONE; } enum iris_buffer_type v4l2_type_to_driver(u32 type) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c index 2850fd5..00e598d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.c @@ -668,6 +668,59 @@ int iris_hfi_set_property(struct iris_inst *inst, return ret; } +int iris_hfi_set_ir_period(struct iris_inst *inst, + u32 packet_type, u32 flag, u32 plane, u32 payload_type, + void *payload, u32 payload_size) +{ + u32 sync_frame_req = 0; + struct iris_core *core; + int ret; + + core = inst->core; + + mutex_lock(&core->lock); + + ret = hfi_create_header(inst->packet, inst->packet_size, + inst->session_id, core->header_id++); + if (ret) + goto exit; + + if (!inst->ir_enabled) { + inst->ir_enabled = ((*(u32 *)payload > 0) ? true : false); + if (inst->ir_enabled && inst->vb2q_dst->streaming) { + sync_frame_req = HFI_SYNC_FRAME_REQUEST_WITH_PREFIX_SEQ_HDR; + ret = hfi_create_packet(inst->packet, inst->packet_size, + HFI_PROP_REQUEST_SYNC_FRAME, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32_ENUM, + HFI_PORT_BITSTREAM, + core->packet_id++, + &sync_frame_req, + sizeof(u32)); + if (ret) + goto exit; + } + } + + ret = hfi_create_packet(inst->packet, inst->packet_size, + packet_type, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + plane, + core->packet_id++, + payload, + sizeof(u32)); + if (ret) + goto exit; + + ret = iris_hfi_queue_cmd_write(inst->core, inst->packet); + +exit: + mutex_unlock(&core->lock); + + return ret; +} + int iris_hfi_queue_buffer(struct iris_inst *inst, struct iris_buffer *buffer) { diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h index 465bfc5..95e0523 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi.h @@ -40,9 +40,8 @@ int prepare_pc(struct iris_core *core); irqreturn_t iris_hfi_isr(int irq, void *data); irqreturn_t iris_hfi_isr_handler(int irq, void *data); -int iris_hfi_queue_buffer(struct iris_inst *inst, - struct iris_buffer *buffer); -int iris_hfi_release_buffer(struct iris_inst *inst, - struct iris_buffer *buffer); +int iris_hfi_set_ir_period(struct iris_inst *inst, + u32 packet_type, u32 flag, u32 plane, u32 payload_type, + void *payload, u32 payload_size); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c index fab206cf..dd27fa4 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.c @@ -364,8 +364,8 @@ int get_hfi_buffer(struct iris_buffer *buffer, struct hfi_buffer *buf) return 0; } -static int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id, - u32 header_id) +int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id, + u32 header_id) { struct hfi_header *hdr = (struct hfi_header *)packet; @@ -382,9 +382,9 @@ static int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id, return 0; } -static int hfi_create_packet(u8 *packet, u32 packet_size, u32 pkt_type, - u32 pkt_flags, u32 payload_type, u32 port, - u32 packet_id, void *payload, u32 payload_size) +int hfi_create_packet(u8 *packet, u32 packet_size, u32 pkt_type, + u32 pkt_flags, u32 payload_type, u32 port, + u32 packet_id, void *payload, u32 payload_size) { struct hfi_header *hdr; struct hfi_packet *pkt; diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h index 849b585..82148b7 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_packet.h @@ -108,5 +108,10 @@ int hfi_packet_sys_interframe_powercollapse(struct iris_core *core, u8 *pkt, u32 pkt_size); int hfi_packet_sys_pc_prep(struct iris_core *core, u8 *pkt, u32 pkt_size); +int hfi_create_header(u8 *packet, u32 packet_size, u32 session_id, + u32 header_id); +int hfi_create_packet(u8 *packet, u32 packet_size, u32 pkt_type, + u32 pkt_flags, u32 payload_type, u32 port, + u32 packet_id, void *payload, u32 payload_size); #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c index a4d5d9c..1b667a5 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c +++ b/drivers/media/platform/qcom/vcodec/iris/iris_hfi_response.c @@ -758,6 +758,12 @@ static int handle_session_property(struct iris_inst *inst, case HFI_PROP_PICTURE_TYPE: inst->hfi_frame_info.picture_type = payload_ptr[0]; break; + case HFI_PROP_CABAC_SESSION: + if (payload_ptr[0] == 1) + inst->cap[ENTROPY_MODE].value = V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC; + else + inst->cap[ENTROPY_MODE].value = V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC; + break; case HFI_PROP_DPB_LIST: ret = handle_dpb_list_property(inst, pkt); if (ret) diff --git a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h index 3c85e78..70f4c7d 100644 --- a/drivers/media/platform/qcom/vcodec/iris/iris_instance.h +++ b/drivers/media/platform/qcom/vcodec/iris/iris_instance.h @@ -52,6 +52,9 @@ * @power: structure of power info * @bus_data: structure of bus data * @input_timer_list: list head of input timer + * @ir_enabled: boolean for intra refresh + * @hfi_rc_type: rate control type + * @hfi_layer_type: type of HFI layer encoding */ struct iris_inst { @@ -90,6 +93,9 @@ struct iris_inst { struct iris_inst_power power; struct bus_vote_data bus_data; struct list_head input_timer_list; + bool ir_enabled; + u32 hfi_rc_type; + u32 hfi_layer_type; }; #endif diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_common.h b/drivers/media/platform/qcom/vcodec/iris/platform_common.h index 81de610..443894c 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_common.h +++ b/drivers/media/platform/qcom/vcodec/iris/platform_common.h @@ -20,9 +20,29 @@ struct iris_inst; #define BIT_DEPTH_8 (8 << 16 | 8) #define BIT_DEPTH_10 (10 << 16 | 10) -#define CODED_FRAMES_PROGRESSIVE 0x0 -#define CODED_FRAMES_INTERLACE 0x1 -#define MAX_NUM_CHILD 10 +#define CODED_FRAMES_PROGRESSIVE 0x0 +#define CODED_FRAMES_INTERLACE 0x1 +#define MAX_NUM_CHILD 10 +#define MAX_ENH_LAYER_HB 3 +#define MAX_HEVC_VBR_ENH_LAYER_SLIDING_WINDOW 5 +#define MAX_HEVC_NON_VBR_ENH_LAYER_SLIDING_WINDOW 3 +#define MAX_AVC_ENH_LAYER_SLIDING_WINDOW 3 +#define MAX_AVC_ENH_LAYER_HYBRID_HP 5 +#define MAX_SLICES_FRAME_RATE 60 +#define MAX_MB_SLICE_WIDTH 4096 +#define MAX_MB_SLICE_HEIGHT 2160 +#define MAX_BYTES_SLICE_WIDTH 1920 +#define MAX_BYTES_SLICE_HEIGHT 1088 +#define MIN_HEVC_SLICE_WIDTH 384 +#define MIN_AVC_SLICE_WIDTH 192 +#define MIN_SLICE_HEIGHT 128 +#define MAX_SLICES_PER_FRAME 10 +#define MIN_QP_8BIT 1 +#define MIN_SLICE_BYTE_SIZE 512 +#define MAX_SLICE_BYTE_SIZE ((MAX_BITRATE) >> 3) +#define MAX_SLICE_MB_SIZE (((4096 + 15) >> 4) * ((2304 + 15) >> 4)) +#define INVALID_DEFAULT_MARK_OR_USE_LTR -1 +#define MAX_SUPPORTED_MIN_QUALITY 70 #define DEFAULT_MAX_HOST_BUF_COUNT 64 #define DEFAULT_MAX_HOST_BURST_BUF_COUNT 256 @@ -128,12 +148,17 @@ enum plat_inst_cap_type { PIX_FMTS, MBPF, QUEUED_RATE, + FRAME_RATE, + OPERATING_RATE, MB_CYCLES_VSP, MB_CYCLES_VPP, MB_CYCLES_LP, MB_CYCLES_FW, MB_CYCLES_FW_VPP, NUM_COMV, + ENTROPY_MODE, + BASELAYER_PRIORITY, + IR_TYPE, PROFILE, LEVEL, HEVC_TIER, @@ -148,6 +173,48 @@ enum plat_inst_cap_type { BIT_DEPTH, DEFAULT_HEADER, RAP_FRAME, + MIN_FRAME_QP, + MAX_FRAME_QP, + B_FRAME, + I_FRAME_QP, + P_FRAME_QP, + B_FRAME_QP, + BIT_RATE, + PEAK_BITRATE, + BITRATE_MODE, + FRAME_SKIP_MODE, + FRAME_RC_ENABLE, + GOP_SIZE, + MIN_QUALITY, + IR_PERIOD, + LTR_COUNT, + USE_LTR, + MARK_LTR, + I_FRAME_MIN_QP, + P_FRAME_MIN_QP, + B_FRAME_MIN_QP, + I_FRAME_MAX_QP, + P_FRAME_MAX_QP, + B_FRAME_MAX_QP, + LAYER_TYPE, + LAYER_ENABLE, + L0_BR, + L1_BR, + L2_BR, + L3_BR, + L4_BR, + L5_BR, + ENH_LAYER_COUNT, + TRANSFORM_8X8, + SLICE_MODE, + SLICE_MAX_MB, + SLICE_MAX_BYTES, + HFLIP, + VFLIP, + ROTATION, + HEADER_MODE, + PREPEND_SPSPPS_TO_IDR, + REQUEST_I_FRAME, INST_CAP_MAX, }; @@ -164,6 +231,7 @@ enum plat_inst_cap_flags { struct plat_inst_cap { enum plat_inst_cap_type cap_id; + enum domain_type domain; enum codec_type codec; s32 min; s32 max; diff --git a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c index 7ae9715..6d5192a 100644 --- a/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c +++ b/drivers/media/platform/qcom/vcodec/iris/platform_sm8550.c @@ -13,11 +13,19 @@ #include "platform_common.h" #include "resources.h" -#define CODECS_ALL (H264 | HEVC | VP9) - -#define DEFAULT_FPS 30 -#define MINIMUM_FPS 1 -#define MAXIMUM_FPS 480 +#define CODECS_ALL (H264 | HEVC | VP9) +#define ENC ENCODER +#define DEC DECODER + +#define DEFAULT_FPS 30 +#define MINIMUM_FPS 1 +#define MAXIMUM_FPS 480 +#define MAX_BITRATE 245000000 +#define DEFAULT_BITRATE 20000000 +#define MAX_LTR_FRAME_COUNT 2 +#define MAX_BASE_LAYER_PRIORITY_ID 63 +#define MAX_QP 51 +#define DEFAULT_QP 20 static struct codec_info codec_data_sm8550[] = { { @@ -68,21 +76,37 @@ static struct plat_core_cap core_data_sm8550[] = { }; static struct plat_inst_cap instance_cap_data_sm8550[] = { - {FRAME_WIDTH, CODECS_ALL, 96, 8192, 1, 1920}, + {FRAME_WIDTH, DEC, CODECS_ALL, 96, 8192, 1, 1920}, + + {FRAME_WIDTH, DEC, VP9, 96, 4096, 1, 1920}, + + {FRAME_WIDTH, ENC, CODECS_ALL, 128, 8192, 1, 1920}, - {FRAME_WIDTH, VP9, 96, 4096, 1, 1920}, + {FRAME_WIDTH, ENC, HEVC, 96, 8192, 1, 1920}, - {FRAME_HEIGHT, CODECS_ALL, 96, 8192, 1, 1080}, + {FRAME_HEIGHT, DEC, CODECS_ALL, 96, 8192, 1, 1080}, - {FRAME_HEIGHT, VP9, 96, 4096, 1, 1080}, + {FRAME_HEIGHT, DEC, VP9, 96, 4096, 1, 1080}, - {PIX_FMTS, H264, + {FRAME_HEIGHT, ENC, CODECS_ALL, 128, 8192, 1, 1080}, + + {FRAME_HEIGHT, ENC, HEVC, 96, 8192, 1, 1080}, + + {PIX_FMTS, DEC, H264, FMT_NV12, FMT_NV12C, FMT_NV12 | FMT_NV21 | FMT_NV12C, FMT_NV12C}, - {PIX_FMTS, HEVC, + {PIX_FMTS, ENC, H264, + FMT_NV12, + FMT_NV12C, + FMT_NV12 | FMT_NV21 | FMT_NV12C, + FMT_NV12C, + 0, 0, + CAP_FLAG_NONE}, + + {PIX_FMTS, DEC, HEVC, FMT_NV12, FMT_TP10C, FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C, @@ -91,37 +115,73 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { CAP_FLAG_NONE, {PROFILE}}, - {PIX_FMTS, VP9, + {PIX_FMTS, ENC, HEVC, + FMT_NV12, + FMT_TP10C, + FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C, + FMT_NV12C, + 0, 0, + CAP_FLAG_NONE, + {PROFILE, MIN_QUALITY, IR_PERIOD, LTR_COUNT}}, + + {PIX_FMTS, DEC, VP9, FMT_NV12, FMT_TP10C, FMT_NV12 | FMT_NV21 | FMT_NV12C | FMT_TP10C, FMT_NV12C}, - {MBPF, CODECS_ALL, 36, 138240, 1, 138240}, + {MBPF, DEC, CODECS_ALL, 36, 138240, 1, 138240}, /* (4096 * 2304) / 256 */ - {MBPF, VP9, 36, 36864, 1, 36864}, + {MBPF, DEC, VP9, 36, 36864, 1, 36864}, + + /* (8192 * 4320) / 256 */ + {MBPF, ENC, CODECS_ALL, 64, 138240, 1, 138240}, + + {MBPF, ENC, HEVC, 36, 138240, 1, 138240}, - {QUEUED_RATE, CODECS_ALL, + {QUEUED_RATE, DEC | ENC, CODECS_ALL, (MINIMUM_FPS << 16), INT_MAX, 1, (DEFAULT_FPS << 16)}, - {MB_CYCLES_VSP, CODECS_ALL, 25, 25, 1, 25}, + {FRAME_RATE, ENC, CODECS_ALL, + (MINIMUM_FPS << 16), (MAXIMUM_FPS << 16), + 1, (DEFAULT_FPS << 16), + 0, + HFI_PROP_FRAME_RATE, + CAP_FLAG_OUTPUT_PORT, + {0}, + NULL, + set_q16}, + + {OPERATING_RATE, ENC, CODECS_ALL, + (MINIMUM_FPS << 16), (MAXIMUM_FPS << 16), + 1, (DEFAULT_FPS << 16)}, + + {MB_CYCLES_VSP, DEC, CODECS_ALL, 25, 25, 1, 25}, + + {MB_CYCLES_VSP, DEC, VP9, 60, 60, 1, 60}, + + {MB_CYCLES_VSP, ENC, CODECS_ALL, 25, 25, 1, 25}, - {MB_CYCLES_VSP, VP9, 60, 60, 1, 60}, + {MB_CYCLES_VPP, DEC, CODECS_ALL, 200, 200, 1, 200}, - {MB_CYCLES_VPP, CODECS_ALL, 200, 200, 1, 200}, + {MB_CYCLES_VPP, ENC, CODECS_ALL, 675, 675, 1, 675}, - {MB_CYCLES_LP, CODECS_ALL, 200, 200, 1, 200}, + {MB_CYCLES_LP, DEC, CODECS_ALL, 200, 200, 1, 200}, - {MB_CYCLES_FW, CODECS_ALL, 489583, 489583, 1, 489583}, + {MB_CYCLES_LP, ENC, CODECS_ALL, 320, 320, 1, 320}, - {MB_CYCLES_FW_VPP, CODECS_ALL, 66234, 66234, 1, 66234}, + {MB_CYCLES_FW, DEC | ENC, CODECS_ALL, 489583, 489583, 1, 489583}, - {NUM_COMV, CODECS_ALL, + {MB_CYCLES_FW_VPP, DEC, CODECS_ALL, 66234, 66234, 1, 66234}, + + {MB_CYCLES_FW_VPP, ENC, CODECS_ALL, 48405, 48405, 1, 48405}, + + {NUM_COMV, DEC, CODECS_ALL, 0, INT_MAX, 1, 0}, - {PROFILE, H264, + {PROFILE, DEC, H264, V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH, BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) | @@ -137,7 +197,23 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_u32_enum}, - {PROFILE, HEVC, + {PROFILE, ENC, H264, + V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, + V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH, + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) | + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH) | + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) | + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) | + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_HIGH), + V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + V4L2_CID_MPEG_VIDEO_H264_PROFILE, + HFI_PROP_PROFILE, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {ENTROPY_MODE, TRANSFORM_8X8}, + NULL, + set_u32_enum}, + + {PROFILE, ENC | DEC, HEVC, V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN, V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_10, BIT(V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN) | @@ -151,7 +227,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { adjust_profile, set_u32_enum}, - {PROFILE, VP9, + {PROFILE, DEC, VP9, V4L2_MPEG_VIDEO_VP9_PROFILE_0, V4L2_MPEG_VIDEO_VP9_PROFILE_2, BIT(V4L2_MPEG_VIDEO_VP9_PROFILE_0) | @@ -164,7 +240,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_u32_enum}, - {LEVEL, H264, + {LEVEL, DEC, H264, V4L2_MPEG_VIDEO_H264_LEVEL_1_0, V4L2_MPEG_VIDEO_H264_LEVEL_6_2, BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_0) | @@ -195,7 +271,36 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_u32_enum}, - {LEVEL, HEVC, + {LEVEL, ENC, H264, + V4L2_MPEG_VIDEO_H264_LEVEL_1_0, + V4L2_MPEG_VIDEO_H264_LEVEL_6_0, + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1B) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_3) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_0), + V4L2_MPEG_VIDEO_H264_LEVEL_5_0, + V4L2_CID_MPEG_VIDEO_H264_LEVEL, + HFI_PROP_LEVEL, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {0}, + NULL, + set_level}, + + {LEVEL, DEC, HEVC, V4L2_MPEG_VIDEO_HEVC_LEVEL_1, V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2, BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_1) | @@ -219,7 +324,31 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_u32_enum}, - {LEVEL, VP9, + {LEVEL, ENC, HEVC, + V4L2_MPEG_VIDEO_HEVC_LEVEL_1, + V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2, + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_2) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_2_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_3) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_3_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_4) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_4_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_5_2) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6_1) | + BIT(V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2), + V4L2_MPEG_VIDEO_HEVC_LEVEL_5, + V4L2_CID_MPEG_VIDEO_HEVC_LEVEL, + HFI_PROP_LEVEL, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {0}, + NULL, + set_level}, + + {LEVEL, DEC, VP9, V4L2_MPEG_VIDEO_VP9_LEVEL_1_0, V4L2_MPEG_VIDEO_VP9_LEVEL_6_0, BIT(V4L2_MPEG_VIDEO_VP9_LEVEL_1_0) | @@ -242,7 +371,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_u32_enum}, - {HEVC_TIER, HEVC, + {HEVC_TIER, DEC | ENC, HEVC, V4L2_MPEG_VIDEO_HEVC_TIER_MAIN, V4L2_MPEG_VIDEO_HEVC_TIER_HIGH, BIT(V4L2_MPEG_VIDEO_HEVC_TIER_MAIN) | @@ -255,7 +384,566 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_u32_enum}, - {DISPLAY_DELAY_ENABLE, CODECS_ALL, + {HFLIP, ENC, CODECS_ALL, + 0, 1, 1, 0, + V4L2_CID_HFLIP, + HFI_PROP_FLIP, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_flip}, + + {VFLIP, ENC, CODECS_ALL, + 0, 1, 1, 0, + V4L2_CID_VFLIP, + HFI_PROP_FLIP, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_flip}, + + {ROTATION, ENC, CODECS_ALL, + 0, 270, 90, 0, + V4L2_CID_ROTATE, + HFI_PROP_ROTATION, + CAP_FLAG_OUTPUT_PORT, + {0}, + NULL, + set_rotation}, + + {HEADER_MODE, ENC, CODECS_ALL, + V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE, + V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME, + BIT(V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE) | + BIT(V4L2_MPEG_VIDEO_HEADER_MODE_JOINED_WITH_1ST_FRAME), + V4L2_MPEG_VIDEO_HEADER_MODE_SEPARATE, + V4L2_CID_MPEG_VIDEO_HEADER_MODE, + HFI_PROP_SEQ_HEADER_MODE, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {0}, + NULL, + set_header_mode}, + + {PREPEND_SPSPPS_TO_IDR, ENC, CODECS_ALL, + 0, 1, 1, 0, + V4L2_CID_MPEG_VIDEO_PREPEND_SPSPPS_TO_IDR}, + + {REQUEST_I_FRAME, ENC, H264 | HEVC, + 0, 0, 0, 0, + V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME, + HFI_PROP_REQUEST_SYNC_FRAME, + CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_req_sync_frame}, + + {BIT_RATE, ENC, H264 | HEVC, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_BITRATE, + HFI_PROP_TOTAL_BITRATE, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {PEAK_BITRATE, L0_BR}, + adjust_bitrate, + set_bitrate}, + + {PEAK_BITRATE, ENC, H264 | HEVC, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_BITRATE_PEAK, + HFI_PROP_TOTAL_PEAK_BITRATE, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + adjust_peak_bitrate, + set_peak_bitrate}, + + {BITRATE_MODE, ENC, H264 | HEVC, + V4L2_MPEG_VIDEO_BITRATE_MODE_VBR, + V4L2_MPEG_VIDEO_BITRATE_MODE_CBR, + BIT(V4L2_MPEG_VIDEO_BITRATE_MODE_VBR) | + BIT(V4L2_MPEG_VIDEO_BITRATE_MODE_CBR), + V4L2_MPEG_VIDEO_BITRATE_MODE_VBR, + V4L2_CID_MPEG_VIDEO_BITRATE_MODE, + HFI_PROP_RATE_CONTROL, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {LTR_COUNT, IR_PERIOD, I_FRAME_QP, P_FRAME_QP, + B_FRAME_QP, ENH_LAYER_COUNT, BIT_RATE, + MIN_QUALITY, PEAK_BITRATE, SLICE_MODE}, + adjust_bitrate_mode, + set_u32_enum}, + + {FRAME_SKIP_MODE, ENC, H264 | HEVC, + V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_DISABLED, + V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_BUF_LIMIT, + BIT(V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_DISABLED) | + BIT(V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_LEVEL_LIMIT) | + BIT(V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_BUF_LIMIT), + V4L2_MPEG_VIDEO_FRAME_SKIP_MODE_DISABLED, + V4L2_CID_MPEG_VIDEO_FRAME_SKIP_MODE, + 0, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + + {FRAME_RC_ENABLE, ENC, H264 | HEVC, + 0, 1, 1, 1, + V4L2_CID_MPEG_VIDEO_FRAME_RC_ENABLE}, + + {GOP_SIZE, ENC, CODECS_ALL, + 0, INT_MAX, 1, 2 * DEFAULT_FPS - 1, + V4L2_CID_MPEG_VIDEO_GOP_SIZE, + HFI_PROP_MAX_GOP_FRAMES, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + adjust_gop_size, + set_gop_size}, + + {B_FRAME, ENC, H264 | HEVC, + 0, 7, 1, 0, + V4L2_CID_MPEG_VIDEO_B_FRAMES, + HFI_PROP_MAX_B_FRAMES, + CAP_FLAG_OUTPUT_PORT, + {0}, + adjust_b_frame, + set_u32}, + + {LTR_COUNT, ENC, H264 | HEVC, + 0, MAX_LTR_FRAME_COUNT, 1, 0, + V4L2_CID_MPEG_VIDEO_LTR_COUNT, + HFI_PROP_LTR_COUNT, + CAP_FLAG_OUTPUT_PORT, + {0}, + adjust_ltr_count, + set_u32}, + + {USE_LTR, ENC, H264 | HEVC, + 0, + ((1 << MAX_LTR_FRAME_COUNT) - 1), + 0, 0, + V4L2_CID_MPEG_VIDEO_USE_LTR_FRAMES, + HFI_PROP_LTR_USE, + CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + adjust_use_ltr, + set_use_and_mark_ltr}, + + {MARK_LTR, ENC, H264 | HEVC, + INVALID_DEFAULT_MARK_OR_USE_LTR, + (MAX_LTR_FRAME_COUNT - 1), + 1, INVALID_DEFAULT_MARK_OR_USE_LTR, + V4L2_CID_MPEG_VIDEO_FRAME_LTR_INDEX, + HFI_PROP_LTR_MARK, + CAP_FLAG_INPUT_PORT | CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + adjust_mark_ltr, + set_use_and_mark_ltr}, + + {BASELAYER_PRIORITY, ENC, H264, + 0, MAX_BASE_LAYER_PRIORITY_ID, 1, 0, + V4L2_CID_MPEG_VIDEO_BASELAYER_PRIORITY_ID, + HFI_PROP_BASELAYER_PRIORITYID, + CAP_FLAG_OUTPUT_PORT}, + + {IR_TYPE, ENC, H264 | HEVC, + V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM, + V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC, + BIT(V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM) | + BIT(V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_CYCLIC), + V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE_RANDOM, + V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD_TYPE, + 0, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU}, + + {IR_PERIOD, ENC, H264 | HEVC, + 0, INT_MAX, 1, 0, + V4L2_CID_MPEG_VIDEO_INTRA_REFRESH_PERIOD, + 0, + CAP_FLAG_INPUT_PORT | CAP_FLAG_OUTPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + adjust_ir_period, + set_ir_period}, + + {MIN_QUALITY, ENC, H264 | HEVC, + 0, MAX_SUPPORTED_MIN_QUALITY, 70, MAX_SUPPORTED_MIN_QUALITY, + 0, + HFI_PROP_MAINTAIN_MIN_QUALITY, + CAP_FLAG_OUTPUT_PORT, + {0}, + adjust_min_quality, + set_u32}, + + {MIN_FRAME_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT, + V4L2_CID_MPEG_VIDEO_H264_MIN_QP, + HFI_PROP_MIN_QP_PACKED, + CAP_FLAG_OUTPUT_PORT, + {0}, + NULL, + set_min_qp}, + + {MIN_FRAME_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT, + V4L2_CID_MPEG_VIDEO_HEVC_MIN_QP, + HFI_PROP_MIN_QP_PACKED, + CAP_FLAG_OUTPUT_PORT, + {0}, + NULL, + set_min_qp}, + + {MAX_FRAME_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, MAX_QP, + V4L2_CID_MPEG_VIDEO_H264_MAX_QP, + HFI_PROP_MAX_QP_PACKED, + CAP_FLAG_OUTPUT_PORT, + {0}, + NULL, + set_max_qp}, + + {MAX_FRAME_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, MAX_QP, + V4L2_CID_MPEG_VIDEO_HEVC_MAX_QP, + HFI_PROP_MAX_QP_PACKED, + CAP_FLAG_OUTPUT_PORT, + {0}, + NULL, + set_max_qp}, + + {I_FRAME_MIN_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT, + V4L2_CID_MPEG_VIDEO_H264_I_FRAME_MIN_QP}, + + {I_FRAME_MIN_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT, + V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_MIN_QP}, + + {P_FRAME_MIN_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT, + V4L2_CID_MPEG_VIDEO_H264_P_FRAME_MIN_QP}, + + {P_FRAME_MIN_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT, + V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_MIN_QP}, + + {B_FRAME_MIN_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT, + V4L2_CID_MPEG_VIDEO_H264_B_FRAME_MIN_QP}, + + {B_FRAME_MIN_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, MIN_QP_8BIT, + V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_MIN_QP}, + + {I_FRAME_MAX_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, MAX_QP, + V4L2_CID_MPEG_VIDEO_H264_I_FRAME_MAX_QP}, + + {I_FRAME_MAX_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, MAX_QP, + V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_MAX_QP}, + + {P_FRAME_MAX_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, MAX_QP, + V4L2_CID_MPEG_VIDEO_H264_P_FRAME_MAX_QP}, + + {P_FRAME_MAX_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, MAX_QP, + V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_MAX_QP}, + + {B_FRAME_MAX_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, MAX_QP, + V4L2_CID_MPEG_VIDEO_H264_B_FRAME_MAX_QP}, + + {B_FRAME_MAX_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, MAX_QP, + V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_MAX_QP}, + + {I_FRAME_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP, + V4L2_CID_MPEG_VIDEO_HEVC_I_FRAME_QP, + HFI_PROP_QP_PACKED, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_frame_qp}, + + {I_FRAME_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP, + V4L2_CID_MPEG_VIDEO_H264_I_FRAME_QP, + HFI_PROP_QP_PACKED, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_frame_qp}, + + {P_FRAME_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP, + V4L2_CID_MPEG_VIDEO_HEVC_P_FRAME_QP, + HFI_PROP_QP_PACKED, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_frame_qp}, + + {P_FRAME_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP, + V4L2_CID_MPEG_VIDEO_H264_P_FRAME_QP, + HFI_PROP_QP_PACKED, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_frame_qp}, + + {B_FRAME_QP, ENC, HEVC, + MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP, + V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP, + HFI_PROP_QP_PACKED, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_frame_qp}, + + {B_FRAME_QP, ENC, H264, + MIN_QP_8BIT, MAX_QP, 1, DEFAULT_QP, + V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP, + HFI_PROP_QP_PACKED, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + NULL, + set_frame_qp}, + + {LAYER_TYPE, ENC, HEVC, + V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B, + V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P, + BIT(V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_B) | + BIT(V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P), + V4L2_MPEG_VIDEO_HEVC_HIERARCHICAL_CODING_P, + V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_TYPE, + HFI_PROP_LAYER_ENCODING_TYPE, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {LTR_COUNT}}, + + {LAYER_TYPE, ENC, H264, + V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B, + V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P, + BIT(V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_B) | + BIT(V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P), + V4L2_MPEG_VIDEO_H264_HIERARCHICAL_CODING_P, + V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_TYPE, + HFI_PROP_LAYER_ENCODING_TYPE, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {LTR_COUNT}}, + + {LAYER_ENABLE, ENC, H264, + 0, 1, 1, 0, + V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING, + HFI_PROP_LAYER_ENCODING_TYPE, + CAP_FLAG_OUTPUT_PORT}, + + {LAYER_ENABLE, ENC, HEVC, + 0, 1, 1, 0, + 0, + 0, + CAP_FLAG_OUTPUT_PORT}, + + {ENH_LAYER_COUNT, ENC, HEVC, + 0, 5, 1, 0, + V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_LAYER, + HFI_PROP_LAYER_COUNT, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {GOP_SIZE, B_FRAME, BIT_RATE, MIN_QUALITY, SLICE_MODE, + LTR_COUNT}, + adjust_layer_count, + set_layer_count_and_type}, + + {ENH_LAYER_COUNT, ENC, H264, + 0, 5, 1, 0, + V4L2_CID_MPEG_VIDEO_H264_HIERARCHICAL_CODING_LAYER, + HFI_PROP_LAYER_COUNT, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {GOP_SIZE, B_FRAME, BIT_RATE, MIN_QUALITY, SLICE_MODE, + LTR_COUNT}, + adjust_layer_count, + set_layer_count_and_type}, + + {L0_BR, ENC, H264, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L0_BR, + HFI_PROP_BITRATE_LAYER1, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L1_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L0_BR, ENC, HEVC, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L0_BR, + HFI_PROP_BITRATE_LAYER1, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L1_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L1_BR, ENC, H264, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L1_BR, + HFI_PROP_BITRATE_LAYER2, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L2_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L1_BR, ENC, HEVC, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L1_BR, + HFI_PROP_BITRATE_LAYER2, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L2_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L2_BR, ENC, H264, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L2_BR, + HFI_PROP_BITRATE_LAYER3, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L3_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L2_BR, ENC, HEVC, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L2_BR, + HFI_PROP_BITRATE_LAYER3, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L3_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L3_BR, ENC, H264, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L3_BR, + HFI_PROP_BITRATE_LAYER4, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L4_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L3_BR, ENC, HEVC, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L3_BR, + HFI_PROP_BITRATE_LAYER4, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L4_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L4_BR, ENC, H264, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L4_BR, + HFI_PROP_BITRATE_LAYER5, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L5_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L4_BR, ENC, HEVC, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L4_BR, + HFI_PROP_BITRATE_LAYER5, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {L5_BR}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L5_BR, ENC, H264, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_H264_HIER_CODING_L5_BR, + HFI_PROP_BITRATE_LAYER6, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {L5_BR, ENC, HEVC, + 1, MAX_BITRATE, 1, DEFAULT_BITRATE, + V4L2_CID_MPEG_VIDEO_HEVC_HIER_CODING_L5_BR, + HFI_PROP_BITRATE_LAYER6, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_INPUT_PORT | + CAP_FLAG_DYNAMIC_ALLOWED, + {0}, + adjust_layer_bitrate, + set_layer_bitrate}, + + {ENTROPY_MODE, ENC, H264, + V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC, + V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC, + BIT(V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CAVLC) | + BIT(V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC), + V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC, + V4L2_CID_MPEG_VIDEO_H264_ENTROPY_MODE, + HFI_PROP_CABAC_SESSION, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {BIT_RATE}, + adjust_entropy_mode, + set_u32}, + + {SLICE_MODE, ENC, H264 | HEVC, + V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE, + V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES, + BIT(V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE) | + BIT(V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_MB) | + BIT(V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_MAX_BYTES), + V4L2_MPEG_VIDEO_MULTI_SLICE_MODE_SINGLE, + V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MODE, + 0, + CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + {STAGE}, + adjust_slice_count, + set_slice_count}, + + {SLICE_MAX_BYTES, ENC, H264 | HEVC, + MIN_SLICE_BYTE_SIZE, MAX_SLICE_BYTE_SIZE, + 1, MIN_SLICE_BYTE_SIZE, + V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_BYTES, + HFI_PROP_MULTI_SLICE_BYTES_COUNT, + CAP_FLAG_OUTPUT_PORT}, + + {SLICE_MAX_MB, ENC, H264 | HEVC, + 1, MAX_SLICE_MB_SIZE, 1, 1, + V4L2_CID_MPEG_VIDEO_MULTI_SLICE_MAX_MB, + HFI_PROP_MULTI_SLICE_MB_COUNT, + CAP_FLAG_OUTPUT_PORT}, + + {TRANSFORM_8X8, ENC, H264, + 0, 1, 1, 1, + V4L2_CID_MPEG_VIDEO_H264_8X8_TRANSFORM, + HFI_PROP_8X8_TRANSFORM, + CAP_FLAG_OUTPUT_PORT, + {0}, + adjust_transform_8x8, + set_u32}, + + {DISPLAY_DELAY_ENABLE, DEC, CODECS_ALL, 0, 1, 1, 0, V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY_ENABLE, HFI_PROP_DECODE_ORDER_OUTPUT, @@ -264,7 +952,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, NULL}, - {DISPLAY_DELAY, CODECS_ALL, + {DISPLAY_DELAY, DEC, CODECS_ALL, 0, 1, 1, 0, V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY, HFI_PROP_DECODE_ORDER_OUTPUT, @@ -273,7 +961,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, NULL}, - {OUTPUT_ORDER, CODECS_ALL, + {OUTPUT_ORDER, DEC, CODECS_ALL, 0, 1, 1, 0, 0, HFI_PROP_DECODE_ORDER_OUTPUT, @@ -282,7 +970,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { adjust_output_order, set_u32}, - {INPUT_BUF_HOST_MAX_COUNT, CODECS_ALL, + {INPUT_BUF_HOST_MAX_COUNT, ENC | DEC, CODECS_ALL, DEFAULT_MAX_HOST_BUF_COUNT, DEFAULT_MAX_HOST_BURST_BUF_COUNT, 1, DEFAULT_MAX_HOST_BUF_COUNT, 0, @@ -292,7 +980,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_u32}, - {STAGE, CODECS_ALL, + {STAGE, ENC | DEC, CODECS_ALL, STAGE_1, STAGE_2, 1, STAGE_2, @@ -303,7 +991,7 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_stage}, - {PIPE, CODECS_ALL, + {PIPE, ENC | DEC, CODECS_ALL, PIPE_1, PIPE_4, 1, PIPE_4, @@ -314,26 +1002,26 @@ static struct plat_inst_cap instance_cap_data_sm8550[] = { NULL, set_pipe}, - {POC, H264, 0, 2, 1, 1, + {POC, DEC, H264, 0, 2, 1, 1, 0, HFI_PROP_PIC_ORDER_CNT_TYPE}, - {CODED_FRAMES, H264 | HEVC, + {CODED_FRAMES, DEC, H264 | HEVC, CODED_FRAMES_PROGRESSIVE, CODED_FRAMES_PROGRESSIVE, 0, CODED_FRAMES_PROGRESSIVE, 0, HFI_PROP_CODED_FRAMES}, - {BIT_DEPTH, CODECS_ALL, BIT_DEPTH_8, BIT_DEPTH_10, 1, BIT_DEPTH_8, + {BIT_DEPTH, DEC, CODECS_ALL, BIT_DEPTH_8, BIT_DEPTH_10, 1, BIT_DEPTH_8, 0, HFI_PROP_LUMA_CHROMA_BIT_DEPTH}, - {DEFAULT_HEADER, CODECS_ALL, + {DEFAULT_HEADER, DEC, CODECS_ALL, 0, 1, 1, 0, 0, HFI_PROP_DEC_DEFAULT_HEADER}, - {RAP_FRAME, CODECS_ALL, + {RAP_FRAME, DEC, CODECS_ALL, 0, 1, 1, 1, 0, HFI_PROP_DEC_START_FROM_RAP_FRAME, diff --git a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c index 5a02f24..58498af 100644 --- a/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c +++ b/drivers/media/platform/qcom/vcodec/iris/vpu_iris3_power.c @@ -42,6 +42,9 @@ u64 iris_calc_freq_iris3(struct iris_inst *inst, u32 data_size) if (inst->codec == VP9) { vsp_cycles = div_u64(vsp_cycles * 170, 100); + } else if (inst->cap[ENTROPY_MODE].value == + V4L2_MPEG_VIDEO_H264_ENTROPY_MODE_CABAC) { + vsp_cycles = div_u64(vsp_cycles * 135, 100); } else { base_cycles = 0; vsp_cycles = div_u64(vsp_cycles, 2);