From patchwork Tue Aug 27 10:05:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 824195 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A3FA619D075; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753194; cv=none; b=K053WPqSTe6KS0qkg00SsFR3Nkt+9ZysMqLR/psGTz2JxL93VjNNTeAg3mElI5ZQykD/jXgR9t8vwLevpdLkuuT69qI5pd1AvlkbQfEKUHCbMFQ0DixqRRXnfvXHNlv2J81svpBXbCjg3NSBXEBeO9f47PhMvXyEfYTU9ns14R4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753194; c=relaxed/simple; bh=B5tNVe1PV3SoTJLIqv+C3vKyv3F2tpDmghdpWiLHW2w=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=gouRstwXNTlGZjPuxu02digsMmj3xLZpD54QZKeo+1Y8Zmv9bYXla2rVFSHAXapT7S5VMVE99QfwfKDA1VZbM+4PfPD6nt6JRiji4TtmyisKF9mF3mZ8y1Z8DgdvKbo496wtNQef740BoZA9OTuJfJjCsMHoxrZOx0F2hBoxPLc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jG2hPbBR; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jG2hPbBR" Received: by smtp.kernel.org (Postfix) with ESMTPS id 2A408C8B7DC; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753194; bh=B5tNVe1PV3SoTJLIqv+C3vKyv3F2tpDmghdpWiLHW2w=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=jG2hPbBR94XA4lwcRzIlxcEuFEHryJWA97hbNSLYy+wKXCRyKZSRheDqfHRjbz9+e 4kP8uKdNee2jZ2UtjQZP/a9wwgbmoh6EFOjDQnzAA+0K5Pg9PextYuVq79vCu3dqtR IrLF4b0DvhyHQdJDOIDuAgmTy0totv808h9rrviroV706pJb8k+lzJMyV3XnRgX7B1 3NyaWhvHA7W0NZolsyKxL84Hg5bBxSdcy44QjlIBv4g7B7VmRGkwJk9HS/EpGhOOmk IofrhACCyN6GbWOpiYPoM+cFp1VSf55fRFu3W7+bYOoysj82HHgL4sJHakrvCR/jRt uQCURyZMUEPLg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16D7CC52D6F; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:26 +0530 Subject: [PATCH v3 01/29] dt-bindings: media: Add sm8550 dt schema Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-1-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=5059; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=u2OAlPLCOO8H0BjK6ZnM9BZA1NpTDVrqxTLA9iybsPE=; b=j8KAkO4eEKvPJOWuVfuZ0Xg1PSc0GrEigitEojwiBZ2X684/AuLkI99rn0TGgTIjvwYR8TyYY c6pM3vKA/v6CzOauxXmk5RK5inWt3gNRXTM2GhGMlUlYdwVvSzWHThE X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal Add a schema description for the iris video encoder/decoder on sm8550. Signed-off-by: Dikshita Agarwal --- .../bindings/media/qcom,sm8550-iris.yaml | 162 +++++++++++++++++++++ 1 file changed, 162 insertions(+) diff --git a/Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml b/Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml new file mode 100644 index 000000000000..a7aa6a6190cf --- /dev/null +++ b/Documentation/devicetree/bindings/media/qcom,sm8550-iris.yaml @@ -0,0 +1,162 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/media/qcom,sm8550-iris.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Qualcomm IRIS video encode and decode accelerators + +maintainers: + - Vikash Garodia + - Dikshita Agarwal + +description: | + The Iris video processing unit is a video encode and decode accelerator + present on Qualcomm platforms. + +allOf: + - $ref: qcom,venus-common.yaml# + +properties: + compatible: + oneOf: + - enum: + - qcom,sm8550-iris + + power-domains: + maxItems: 4 + + power-domain-names: + oneOf: + - items: + - const: venus + - const: vcodec0 + - const: mxc + - const: mmcx + + clocks: + maxItems: 3 + + clock-names: + items: + - const: iface + - const: core + - const: vcodec0_core + + interconnects: + maxItems: 2 + + interconnect-names: + items: + - const: cpu-cfg + - const: video-mem + + resets: + maxItems: 1 + + reset-names: + items: + - const: bus + + iommus: + maxItems: 2 + + dma-coherent: true + + operating-points-v2: true + + opp-table: + type: object + +required: + - compatible + - power-domain-names + - interconnects + - interconnect-names + - resets + - reset-names + - iommus + - dma-coherent + +unevaluatedProperties: false + +examples: + - | + #include + #include + #include + #include + #include + #include + #include + #include + + iris: video-codec@aa00000 { + compatible = "qcom,sm8550-iris"; + + reg = <0x0aa00000 0xf0000>; + interrupts = ; + + power-domains = <&videocc VIDEO_CC_MVS0C_GDSC>, + <&videocc VIDEO_CC_MVS0_GDSC>, + <&rpmhpd RPMHPD_MXC>, + <&rpmhpd RPMHPD_MMCX>; + power-domain-names = "venus", "vcodec0", "mxc", "mmcx"; + + clocks = <&gcc GCC_VIDEO_AXI0_CLK>, + <&videocc VIDEO_CC_MVS0C_CLK>, + <&videocc VIDEO_CC_MVS0_CLK>; + clock-names = "iface", "core", "vcodec0_core"; + + interconnects = <&gem_noc MASTER_APPSS_PROC QCOM_ICC_TAG_ALWAYS + &config_noc SLAVE_VENUS_CFG QCOM_ICC_TAG_ALWAYS>, + <&mmss_noc MASTER_VIDEO QCOM_ICC_TAG_ALWAYS + &mc_virt SLAVE_EBI1 QCOM_ICC_TAG_ALWAYS>; + interconnect-names = "cpu-cfg", "video-mem"; + + memory-region = <&video_mem>; + + resets = <&gcc GCC_VIDEO_AXI0_CLK_ARES>; + reset-names = "bus"; + + iommus = <&apps_smmu 0x1940 0x0000>, + <&apps_smmu 0x1947 0x0000>; + dma-coherent; + + operating-points-v2 = <&iris_opp_table>; + + iris_opp_table: opp-table { + compatible = "operating-points-v2"; + + opp-240000000 { + opp-hz = /bits/ 64 <240000000>; + required-opps = <&rpmhpd_opp_svs>, + <&rpmhpd_opp_low_svs>; + }; + + opp-338000000 { + opp-hz = /bits/ 64 <338000000>; + required-opps = <&rpmhpd_opp_svs>, + <&rpmhpd_opp_svs>; + }; + + opp-366000000 { + opp-hz = /bits/ 64 <366000000>; + required-opps = <&rpmhpd_opp_svs_l1>, + <&rpmhpd_opp_svs_l1>; + }; + + opp-444000000 { + opp-hz = /bits/ 64 <444000000>; + required-opps = <&rpmhpd_opp_turbo>, + <&rpmhpd_opp_turbo>; + }; + + opp-533333334 { + opp-hz = /bits/ 64 <533333334>; + required-opps = <&rpmhpd_opp_turbo_l1>, + <&rpmhpd_opp_turbo_l1>; + }; + }; + }; +... From patchwork Tue Aug 27 10:05:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 822729 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A54CC19D08C; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753194; cv=none; b=XRPfQilRHSURa/Bj0zLm4MjNis6FIHIU9Vfd1qX6NVJ4c/ECuYt2axMmQkIf83jJiPbCeGAgJ0h23vMjvlrUSlpMD4Q5zbwwothhSdQ7iAiD2uPZV2SjsXzCxYwjCst01D0/I+gbVtTtauVmzBZbrzGuRi2PTd3rcmopjplEnso= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753194; c=relaxed/simple; bh=zKj0i2yGql7HRjfp5dYyt3Lo063aPGwvbKN6Pvn8jqs=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Rq/EIfs5I3tm1fAhNYpWVc2IDVI0z9HE7UPE00hvOWSCA4REBH6xqHxsLGXCMtoNfy8S0EHM7Yd5F1WCDxnKoFLQ7pBfbUPqvofpa5jFdu5Jpk+hIAeCwBwnAhPuGfa4pnfXiO+F2boRihtnP++gFyNzjb0WtG1hgdqgtwy9JDA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=R4zppCny; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="R4zppCny" Received: by smtp.kernel.org (Postfix) with ESMTPS id 401D8C8B7DF; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753194; bh=zKj0i2yGql7HRjfp5dYyt3Lo063aPGwvbKN6Pvn8jqs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=R4zppCnyoPv8cbJW/l8AK9sh2KU05AVjugjkiCV8G8v3IMyY9ROYphJ/2+qqo1GJ5 zfOMFyv7JEx15EQQoWO1hQRD+Lc4u1dv4d2Silgo3lwCirvnTX/yf8cNeH46vNPJua fZavLb3FG/gNioqd5/DphwKp6NuuVDM0LgYXSIXQ8PwxmqGiG2LbXnuCuCYEoYbMTk plstywBeflx/jUkiNO2PIsai/GzrlxjjxHm1zj8d0r4g8yeKn3QqqHz0yW5sMfXoNS r6byGjyqIVXv5YAbejt3UEMYVYhZKA5ka8iWj3/LB5cShEpMFneR6FXilTuczLPWaa ZA/Htn7Lg6G/w== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A027C5472E; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:27 +0530 Subject: [PATCH v3 02/29] media: MAINTAINERS: Add Qualcomm Iris video accelerator driver Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-2-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=1103; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=f4t4EfkFMyoaeCFa31G11dRb4r/+CaAlXqjBl8U4iXg=; b=Thrg8vqMvLlIP/qwUlEbz8Qy2jtQ00kzFxAjrXCdXAmLY+3oTHib4cBoRrPi1KjFPUmi2iQe2 8I0Aptv2J4OANis7uKKViTvvNujKSS8weX4qDwRCmbQf/N2gWEpXy+k X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal Add an entry for Iris video decoder accelerator driver. Signed-off-by: Vikash Garodia Signed-off-by: Dikshita Agarwal --- MAINTAINERS | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index 8766f3e5e87e..105e67fca308 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -18898,6 +18898,17 @@ S: Maintained F: Documentation/devicetree/bindings/regulator/vqmmc-ipq4019-regulator.yaml F: drivers/regulator/vqmmc-ipq4019-regulator.c +QUALCOMM IRIS VIDEO ACCELERATOR DRIVER +M: Vikash Garodia +M: Dikshita Agarwal +R: Abhinav Kumar +L: linux-media@vger.kernel.org +L: linux-arm-msm@vger.kernel.org +S: Maintained +T: git git://linuxtv.org/media_tree.git +F: Documentation/devicetree/bindings/media/qcom,*-iris.yaml +F: drivers/media/platform/qcom/iris/ + QUALCOMM NAND CONTROLLER DRIVER M: Manivannan Sadhasivam L: linux-mtd@lists.infradead.org From patchwork Tue Aug 27 10:05:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 822730 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A2A1019D067; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753194; cv=none; b=oTBr59b1PkObC6Rt1V8JNGqDSPwWjDNru6lF6U5hChxZS9VOji/A8gFm5s92clZ/w/6wdmGSoZ6j9vET1Z1vDmfTz3KQMi1O9aO7cq/iovbZ9KYtTijjn0DYT6BDsyvK+yirNvJ4tBFUIJc9Rf65sgJFLCiaV2A3wZKK0JmhH+8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753194; c=relaxed/simple; bh=9/bXUR9vWDeYiPhm6N3/6l7AKwGwD51EVfA2fLYMVmQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ahhemIPsX9pa4uIvHYGa04uCyW1+/q//5BhDil8wFIIBp3wgqvT+1LFB8IUzfAoV/0D21HcyKtRPqfsLM6WB4SaXNRQVcjlgr0BzhaBja0BY2XTJvFgy4q/fSdpvvpLfLYL8HwvH4loJXOA+ArSyWNuh7ToG3NgenNmSG7Maklo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=bZrTnZ5k; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="bZrTnZ5k" Received: by smtp.kernel.org (Postfix) with ESMTPS id 4C381C4AF18; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753194; bh=9/bXUR9vWDeYiPhm6N3/6l7AKwGwD51EVfA2fLYMVmQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=bZrTnZ5kLN58NwjnFzzyi8l17g1UdOywtHNEugK50VvygFejNcg32qt9ZmLvkeBGC aaQrYnUl84JNZ2yDjQi6XyAPMMP8zpLe73HaUycN8jkFSurqsJZlxuKOYqTT5nebDe TGzS/2fMJD55Lb+ieFnDJEXBpgWR4cC3/nmO59ZqVb30ChrsLkT0Nkex9WnSopTbXy DXr6sJr95hFoEZReaVTGR6C8D1xgWTtMOyRkYD8icwdYCyCriEfNPi2CmXtPNaHFXU lY+kY5BxFcjXSoiqt8uRLWMc3hJK42yMO/lROBRCXD8KvLajdI34RACI5QwjyCRNRr 4tuQGeE01dXHw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 404DAC54732; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:28 +0530 Subject: [PATCH v3 03/29] media: iris: add platform driver for iris video device Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-3-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=5969; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=TMJ26H+innc/Bj52EA4UGDFzuDU7dFnrwt6UHLwI580=; b=orECwo3xcGC+nogqmINQ3RXF7HT1Ih2G5JtubsF/sXvxALIoz80udRf+3YLsy4q0O1j+hq1kS 6FxZpRE7PScAbpg1+hsGUhmwmMm1wODH7ghC6OWJf6zKFQz9bNwWJgE X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal In preparation of adding H264 decode functionality, add probe and remove functions to iris video platform driver. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/Kconfig | 1 + drivers/media/platform/qcom/Makefile | 1 + drivers/media/platform/qcom/iris/Kconfig | 9 +++ drivers/media/platform/qcom/iris/Makefile | 3 + drivers/media/platform/qcom/iris/iris_core.h | 29 +++++++ drivers/media/platform/qcom/iris/iris_probe.c | 108 ++++++++++++++++++++++++++ 6 files changed, 151 insertions(+) diff --git a/drivers/media/platform/qcom/Kconfig b/drivers/media/platform/qcom/Kconfig index cc5799b9ea00..4f4d3a68e6e5 100644 --- a/drivers/media/platform/qcom/Kconfig +++ b/drivers/media/platform/qcom/Kconfig @@ -3,4 +3,5 @@ comment "Qualcomm media platform drivers" source "drivers/media/platform/qcom/camss/Kconfig" +source "drivers/media/platform/qcom/iris/Kconfig" source "drivers/media/platform/qcom/venus/Kconfig" diff --git a/drivers/media/platform/qcom/Makefile b/drivers/media/platform/qcom/Makefile index 4f055c396e04..ea2221a202c0 100644 --- a/drivers/media/platform/qcom/Makefile +++ b/drivers/media/platform/qcom/Makefile @@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only obj-y += camss/ +obj-y += iris/ obj-y += venus/ diff --git a/drivers/media/platform/qcom/iris/Kconfig b/drivers/media/platform/qcom/iris/Kconfig new file mode 100644 index 000000000000..a9986aaa448d --- /dev/null +++ b/drivers/media/platform/qcom/iris/Kconfig @@ -0,0 +1,9 @@ +config VIDEO_QCOM_IRIS + tristate "Qualcomm Iris V4L2 decoder driver" + depends on VIDEO_DEV + depends on ARCH_QCOM || COMPILE_TEST + help + This is a V4L2 driver for Qualcomm Iris video accelerator + hardware. It accelerates decoding operations on various + Qualcomm SoCs. + To compile this driver as a module choose m here. diff --git a/drivers/media/platform/qcom/iris/Makefile b/drivers/media/platform/qcom/iris/Makefile new file mode 100644 index 000000000000..913da225486b --- /dev/null +++ b/drivers/media/platform/qcom/iris/Makefile @@ -0,0 +1,3 @@ +iris-objs += iris_probe.o \ + +obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/iris/iris_core.h b/drivers/media/platform/qcom/iris/iris_core.h new file mode 100644 index 000000000000..402f0aaef995 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_core.h @@ -0,0 +1,29 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_CORE_H_ +#define _IRIS_CORE_H_ + +#include + +/** + * struct iris_core - holds core parameters valid for all instances + * + * @dev: reference to device structure + * @reg_base: IO memory base address + * @irq: iris irq + * @v4l2_dev: a holder for v4l2 device structure + * @vdev_dec: iris video device structure for decoder + */ + +struct iris_core { + struct device *dev; + void __iomem *reg_base; + int irq; + struct v4l2_device v4l2_dev; + struct video_device *vdev_dec; +}; + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_probe.c b/drivers/media/platform/qcom/iris/iris_probe.c new file mode 100644 index 000000000000..0a54fdaa1ab5 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_probe.c @@ -0,0 +1,108 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include + +#include "iris_core.h" + +static int iris_register_video_device(struct iris_core *core) +{ + struct video_device *vdev; + int ret; + + vdev = video_device_alloc(); + if (!vdev) + return -ENOMEM; + + strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name)); + vdev->release = video_device_release; + vdev->vfl_dir = VFL_DIR_M2M; + vdev->v4l2_dev = &core->v4l2_dev; + vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING; + + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); + if (ret) + goto err_vdev_release; + + core->vdev_dec = vdev; + video_set_drvdata(vdev, core); + + return 0; + +err_vdev_release: + video_device_release(vdev); + + return ret; +} + +static void iris_remove(struct platform_device *pdev) +{ + struct iris_core *core; + + core = platform_get_drvdata(pdev); + if (!core) + return; + + video_unregister_device(core->vdev_dec); + + v4l2_device_unregister(&core->v4l2_dev); +} + +static int iris_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct iris_core *core; + int ret; + + core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL); + if (!core) + return -ENOMEM; + core->dev = dev; + + core->reg_base = devm_platform_ioremap_resource(pdev, 0); + if (IS_ERR(core->reg_base)) + return PTR_ERR(core->reg_base); + + core->irq = platform_get_irq(pdev, 0); + if (core->irq < 0) + return core->irq; + + ret = v4l2_device_register(dev, &core->v4l2_dev); + if (ret) + return ret; + + ret = iris_register_video_device(core); + if (ret) + goto err_v4l2_unreg; + + platform_set_drvdata(pdev, core); + + return 0; + +err_v4l2_unreg: + v4l2_device_unregister(&core->v4l2_dev); + + return ret; +} + +static const struct of_device_id iris_dt_match[] = { + { .compatible = "qcom,sm8550-iris", }, + { .compatible = "qcom,sm8250-venus", }, + { }, +}; +MODULE_DEVICE_TABLE(of, iris_dt_match); + +static struct platform_driver qcom_iris_driver = { + .probe = iris_probe, + .remove_new = iris_remove, + .driver = { + .name = "qcom-iris", + .of_match_table = iris_dt_match, + }, +}; + +module_platform_driver(qcom_iris_driver); +MODULE_DESCRIPTION("Qualcomm Iris video driver"); +MODULE_LICENSE("GPL"); From patchwork Tue Aug 27 10:05:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 822728 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D090719D8A9; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753194; cv=none; b=mf739Uwiet6Ty0ZWF5EewFG3SrL+a5oDX7fx0LvNwgDHf7/v0ahWdPmYbj05cj2vEgKFk153h37lHofMK8uW2fqhd82M9/GYOHpDbOLqJE9l60dtOX0H99MmHM5Jkt3MDV+adPBDlK4llgGiFH+BeorOaChPVmRsiEWmkLCAQYs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753194; c=relaxed/simple; bh=CrMHvDmHqAZRXl4CoDH64DDwswVtSNt94xvcSPlw5Os=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=bF6MlLVaod/UAgrdFFFZ8sk7ZgjwMkO9+z62F4alHwQy05HF5d8KZhHGRf9RrvF6L8h/AOPpIBjyMMHJa/B2qToudXoB3qZr7iJq8dvMeqG2RHhK4ottcZLcJKmAxbt3f1gaKjWGyEdw4YaZvdO3kLtJRzXzoOlKFH2+gwip3K4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hbf4nSol; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hbf4nSol" Received: by smtp.kernel.org (Postfix) with ESMTPS id 648B3C4AF1D; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753194; bh=CrMHvDmHqAZRXl4CoDH64DDwswVtSNt94xvcSPlw5Os=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=hbf4nSolKhIo5+gK4mpySWNdg8Oa/kjMsn/tZWpzvV2GpNWbSEQONu8oI2Az0UJnw 4XoatvUIs2V3pFx2wEwgA6h8WU2tDQE2BveKVJFAzJiZL8F/Z/yJchJ+yitwhG85d1 gLAdBUzZbbTPDnJoPPcI9hxPx7dqdRoI/IbT+q23raiecy6adZNTvs+xydZBFjgB+w UOMk63EWOeB/SbyH0c8twkSjTBS+yGMkjvoeoP4Lsn0utk9M87vc+F0NZva/1+jDeL Z4wOfWx6wYUGSys7GpK8e5ywT8sa2oBtE7upXpyD6F40XmpkGHxfEUwjXoOpoWHKmj zafyUqBbs9akw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52149C5320E; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:29 +0530 Subject: [PATCH v3 04/29] media: iris: initialize power resources Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-4-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=13266; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=ZUmC1n4wAZ0OYX2YUgQnQPDDKpqiTXXKpfIciQErzzE=; b=GsDFJ2Ze3no5yB/GHG2sYobjiyG4i7WHw7Mg5/Qm5MUbbFl5f3Jq2NWE5SMWIwZLM6SrJSK9U Ra1P5vXUP07Aa1K7DVOo3QZH4vj+v537tGBnTGqvnnErrbMEpBwb+wn X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal Add support for initializing Iris "resources", which are clocks, interconnects, power domains, reset clocks, and clock frequencies used for iris hardware. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/Makefile | 5 +- drivers/media/platform/qcom/iris/iris_core.h | 20 +++ .../platform/qcom/iris/iris_platform_common.h | 36 +++++ .../platform/qcom/iris/iris_platform_sm8250.c | 38 +++++ .../platform/qcom/iris/iris_platform_sm8550.c | 38 +++++ drivers/media/platform/qcom/iris/iris_probe.c | 23 ++- drivers/media/platform/qcom/iris/iris_resources.c | 171 +++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_resources.h | 19 +++ 8 files changed, 347 insertions(+), 3 deletions(-) diff --git a/drivers/media/platform/qcom/iris/Makefile b/drivers/media/platform/qcom/iris/Makefile index 913da225486b..3e8474d064f4 100644 --- a/drivers/media/platform/qcom/iris/Makefile +++ b/drivers/media/platform/qcom/iris/Makefile @@ -1,3 +1,6 @@ -iris-objs += iris_probe.o \ +iris-objs += iris_platform_sm8250.o \ + iris_platform_sm8550.o \ + iris_probe.o \ + iris_resources.o \ obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/iris/iris_core.h b/drivers/media/platform/qcom/iris/iris_core.h index 402f0aaef995..a1afd4387b3a 100644 --- a/drivers/media/platform/qcom/iris/iris_core.h +++ b/drivers/media/platform/qcom/iris/iris_core.h @@ -6,8 +6,12 @@ #ifndef _IRIS_CORE_H_ #define _IRIS_CORE_H_ +#include #include +#include "iris_platform_common.h" +#include "iris_resources.h" + /** * struct iris_core - holds core parameters valid for all instances * @@ -16,6 +20,14 @@ * @irq: iris irq * @v4l2_dev: a holder for v4l2 device structure * @vdev_dec: iris video device structure for decoder + * @icc_tbl: table of iris interconnects + * @icc_count: count of iris interconnects + * @pmdomain_tbl: table of iris power domains + * @opp_pmdomain_tbl: table of opp power domains + * @clock_tbl: table of iris clocks + * @clk_count: count of iris clocks + * @resets: table of iris reset clocks + * @iris_platform_data: a structure for platform data */ struct iris_core { @@ -24,6 +36,14 @@ struct iris_core { int irq; struct v4l2_device v4l2_dev; struct video_device *vdev_dec; + struct icc_bulk_data *icc_tbl; + u32 icc_count; + struct dev_pm_domain_list *pmdomain_tbl; + struct dev_pm_domain_list *opp_pmdomain_tbl; + struct clk_bulk_data *clock_tbl; + u32 clk_count; + struct reset_control_bulk_data *resets; + const struct iris_platform_data *iris_platform_data; }; #endif diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h new file mode 100644 index 000000000000..293fb7e904b0 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h @@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_PLATFORM_COMMON_H_ +#define _IRIS_PLATFORM_COMMON_H_ + +extern struct iris_platform_data sm8550_data; +extern struct iris_platform_data sm8250_data; + +enum platform_clk_type { + IRIS_AXI_CLK, + IRIS_CTRL_CLK, + IRIS_HW_CLK, +}; + +struct platform_clk_data { + enum platform_clk_type clk_type; + const char *clk_name; +}; + +struct iris_platform_data { + const struct icc_info *icc_tbl; + unsigned int icc_tbl_size; + const char * const *pmdomain_tbl; + unsigned int pmdomain_tbl_size; + const char * const *opp_pd_tbl; + unsigned int opp_pd_tbl_size; + const struct platform_clk_data *clk_tbl; + unsigned int clk_tbl_size; + const char * const *clk_rst_tbl; + unsigned int clk_rst_tbl_size; +}; + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c new file mode 100644 index 000000000000..b6f08fc327b5 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c @@ -0,0 +1,38 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_core.h" +#include "iris_platform_common.h" +#include "iris_resources.h" + +static const struct icc_info sm8250_icc_table[] = { + { "cpu-cfg", 1000, 1000 }, + { "video-mem", 1000, 15000000 }, +}; + +static const char * const sm8250_clk_reset_table[] = { "bus", "core" }; + +static const char * const sm8250_pmdomain_table[] = { "venus", "vcodec0" }; + +static const char * const sm8250_opp_pd_table[] = { "mx" }; + +static const struct platform_clk_data sm8250_clk_table[] = { + {IRIS_AXI_CLK, "iface" }, + {IRIS_CTRL_CLK, "core" }, + {IRIS_HW_CLK, "vcodec0_core" }, +}; + +struct iris_platform_data sm8250_data = { + .icc_tbl = sm8250_icc_table, + .icc_tbl_size = ARRAY_SIZE(sm8250_icc_table), + .clk_rst_tbl = sm8250_clk_reset_table, + .clk_rst_tbl_size = ARRAY_SIZE(sm8250_clk_reset_table), + .pmdomain_tbl = sm8250_pmdomain_table, + .pmdomain_tbl_size = ARRAY_SIZE(sm8250_pmdomain_table), + .opp_pd_tbl = sm8250_opp_pd_table, + .opp_pd_tbl_size = ARRAY_SIZE(sm8250_opp_pd_table), + .clk_tbl = sm8250_clk_table, + .clk_tbl_size = ARRAY_SIZE(sm8250_clk_table), +}; diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c new file mode 100644 index 000000000000..f2f9e6f6775f --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c @@ -0,0 +1,38 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_core.h" +#include "iris_platform_common.h" +#include "iris_resources.h" + +static const struct icc_info sm8550_icc_table[] = { + { "cpu-cfg", 1000, 1000 }, + { "video-mem", 1000, 15000000 }, +}; + +static const char * const sm8550_clk_reset_table[] = { "bus" }; + +static const char * const sm8550_pmdomain_table[] = { "venus", "vcodec0" }; + +static const char * const sm8550_opp_pd_table[] = { "mxc", "mmcx" }; + +static const struct platform_clk_data sm8550_clk_table[] = { + {IRIS_AXI_CLK, "iface" }, + {IRIS_CTRL_CLK, "core" }, + {IRIS_HW_CLK, "vcodec0_core" }, +}; + +struct iris_platform_data sm8550_data = { + .icc_tbl = sm8550_icc_table, + .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table), + .clk_rst_tbl = sm8550_clk_reset_table, + .clk_rst_tbl_size = ARRAY_SIZE(sm8550_clk_reset_table), + .pmdomain_tbl = sm8550_pmdomain_table, + .pmdomain_tbl_size = ARRAY_SIZE(sm8550_pmdomain_table), + .opp_pd_tbl = sm8550_opp_pd_table, + .opp_pd_tbl_size = ARRAY_SIZE(sm8550_opp_pd_table), + .clk_tbl = sm8550_clk_table, + .clk_tbl_size = ARRAY_SIZE(sm8550_clk_table), +}; diff --git a/drivers/media/platform/qcom/iris/iris_probe.c b/drivers/media/platform/qcom/iris/iris_probe.c index 0a54fdaa1ab5..2616a31224f9 100644 --- a/drivers/media/platform/qcom/iris/iris_probe.c +++ b/drivers/media/platform/qcom/iris/iris_probe.c @@ -69,6 +69,19 @@ static int iris_probe(struct platform_device *pdev) if (core->irq < 0) return core->irq; + core->iris_platform_data = of_device_get_match_data(core->dev); + if (!core->iris_platform_data) { + ret = -ENODEV; + dev_err_probe(core->dev, ret, "init platform failed\n"); + return ret; + } + + ret = iris_init_resources(core); + if (ret) { + dev_err_probe(core->dev, ret, "init resource failed\n"); + return ret; + } + ret = v4l2_device_register(dev, &core->v4l2_dev); if (ret) return ret; @@ -88,8 +101,14 @@ static int iris_probe(struct platform_device *pdev) } static const struct of_device_id iris_dt_match[] = { - { .compatible = "qcom,sm8550-iris", }, - { .compatible = "qcom,sm8250-venus", }, + { + .compatible = "qcom,sm8550-iris", + .data = &sm8550_data, + }, + { + .compatible = "qcom,sm8250-venus", + .data = &sm8250_data, + }, { }, }; MODULE_DEVICE_TABLE(of, iris_dt_match); diff --git a/drivers/media/platform/qcom/iris/iris_resources.c b/drivers/media/platform/qcom/iris/iris_resources.c new file mode 100644 index 000000000000..57c6f9f3449b --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_resources.c @@ -0,0 +1,171 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include + +#include "iris_core.h" +#include "iris_resources.h" + +static int iris_init_icc(struct iris_core *core) +{ + const struct icc_info *icc_tbl; + u32 ret, i = 0; + + icc_tbl = core->iris_platform_data->icc_tbl; + + core->icc_count = core->iris_platform_data->icc_tbl_size; + core->icc_tbl = devm_kzalloc(core->dev, + sizeof(struct icc_bulk_data) * core->icc_count, + GFP_KERNEL); + if (!core->icc_tbl) + return -ENOMEM; + + for (i = 0; i < core->icc_count; i++) { + core->icc_tbl[i].name = icc_tbl[i].name; + core->icc_tbl[i].avg_bw = icc_tbl[i].bw_min_kbps; + core->icc_tbl[i].peak_bw = 0; + } + + ret = devm_of_icc_bulk_get(core->dev, core->icc_count, core->icc_tbl); + if (ret) + dev_err(core->dev, "failed to get interconnect paths, NoC will stay unconfigured!\n"); + + return ret; +} + +static int iris_pd_get(struct iris_core *core) +{ + int ret; + + struct dev_pm_domain_attach_data iris_pd_data = { + .pd_names = core->iris_platform_data->pmdomain_tbl, + .num_pd_names = core->iris_platform_data->pmdomain_tbl_size, + .pd_flags = PD_FLAG_NO_DEV_LINK, + }; + + ret = devm_pm_domain_attach_list(core->dev, &iris_pd_data, &core->pmdomain_tbl); + if (ret < 0) + return ret; + + return 0; +} + +static int iris_opp_pd_get(struct iris_core *core) +{ + int ret; + + struct dev_pm_domain_attach_data iris_opp_pd_data = { + .pd_names = core->iris_platform_data->opp_pd_tbl, + .num_pd_names = core->iris_platform_data->opp_pd_tbl_size, + .pd_flags = PD_FLAG_DEV_LINK_ON, + }; + + ret = devm_pm_domain_attach_list(core->dev, &iris_opp_pd_data, &core->opp_pmdomain_tbl); + if (ret < 0) + return ret; + + return 0; +} + +static int iris_init_power_domains(struct iris_core *core) +{ + const struct platform_clk_data *clk_tbl; + u32 clk_cnt, i; + int ret; + + ret = iris_pd_get(core); + if (ret) + return ret; + + ret = iris_opp_pd_get(core); + if (ret) + return ret; + + clk_tbl = core->iris_platform_data->clk_tbl; + clk_cnt = core->iris_platform_data->clk_tbl_size; + + for (i = 0; i < clk_cnt; i++) { + if (clk_tbl[i].clk_type == IRIS_HW_CLK) { + ret = devm_pm_opp_set_clkname(core->dev, clk_tbl[i].clk_name); + if (ret) + return ret; + } + } + + ret = devm_pm_opp_of_add_table(core->dev); + if (ret) { + dev_err(core->dev, "failed to add opp table\n"); + return ret; + } + + return ret; +} + +static int iris_init_clocks(struct iris_core *core) +{ + int ret; + + ret = devm_clk_bulk_get_all(core->dev, &core->clock_tbl); + if (ret < 0) { + dev_err(core->dev, "failed to get bulk clock\n"); + return ret; + } + + core->clk_count = ret; + + return 0; +} + +static int iris_init_resets(struct iris_core *core) +{ + const char * const *rst_tbl; + u32 rst_tbl_size; + u32 i = 0, ret; + + rst_tbl = core->iris_platform_data->clk_rst_tbl; + rst_tbl_size = core->iris_platform_data->clk_rst_tbl_size; + + core->resets = devm_kzalloc(core->dev, + sizeof(*core->resets) * rst_tbl_size, + GFP_KERNEL); + if (rst_tbl_size && !core->resets) + return -ENOMEM; + + for (i = 0; i < rst_tbl_size; i++) + core->resets[i].id = rst_tbl[i]; + + ret = devm_reset_control_bulk_get_exclusive(core->dev, rst_tbl_size, core->resets); + if (ret) { + dev_err(core->dev, "failed to get resets\n"); + return ret; + } + + return 0; +} + +int iris_init_resources(struct iris_core *core) +{ + int ret; + + ret = iris_init_icc(core); + if (ret) + return ret; + + ret = iris_init_power_domains(core); + if (ret) + return ret; + + ret = iris_init_clocks(core); + if (ret) + return ret; + + ret = iris_init_resets(core); + + return ret; +} diff --git a/drivers/media/platform/qcom/iris/iris_resources.h b/drivers/media/platform/qcom/iris/iris_resources.h new file mode 100644 index 000000000000..b0217399030a --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_resources.h @@ -0,0 +1,19 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_RESOURCES_H_ +#define _IRIS_RESOURCES_H_ + +struct iris_core; + +struct icc_info { + const char *name; + u32 bw_min_kbps; + u32 bw_max_kbps; +}; + +int iris_init_resources(struct iris_core *core); + +#endif From patchwork Tue Aug 27 10:05:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 824194 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D105419D8BE; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753194; cv=none; b=U+fKNtahXvoqhACBFfxfop3MKYCqxlyRNq17g9XYUmCe/CTh7cxuohhD+nfje4ieRtZXQr9GEMeNmXcTBYCmbaiwtiU0uiEIq+F1UFq0lR2CED2yg7iTU6QM8Ah68OWKB5OTQ00yqluU1meTn1Xt+hCBO+LMAT7tuMquG7Th1N8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753194; c=relaxed/simple; bh=EjMU+8CIbzqeYXltlnVtohsrXYIFzo1X0j7NaKNQXLA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=EcNDm80IC4QesjeLDHMlYs0g4IM+aQpsM9D6o2fGtvhp4hcxhwzEvkJDTFuz1iWTV1qGP8YYRjBp24yzt/TACjrq7EKyWMvACaeid6+YaEpojMWuam+YSVNXyqmzHtVantsjTb/YTGMy7Qcu4PGYHPk3dd4R6mhmgNOUlAaC8ZM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=tlozr1ir; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="tlozr1ir" Received: by smtp.kernel.org (Postfix) with ESMTPS id 70ECCC4AF17; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753194; bh=EjMU+8CIbzqeYXltlnVtohsrXYIFzo1X0j7NaKNQXLA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=tlozr1irN7N153uaKwQwfXQzRtpybSRZGSg+vFYjwjrwNRI4AKBJBiqpHs1wyT64X wa7iHrhbnxWQygwsNA1dcGOejpJBak1eoNskTT9OriBnqE51ils9I5QNX7wYwFuTRU YPJxOsd1lgALh2Waa+y6d+uRQdJEPbzsaB8vxK0l3fTw9bQpXukxRNGxzjlENd3ymE Rb4ANq/U/BvsRfpCYhltfQ1q+yn1H9juX/E1+Ss/5TOlqKvB5TYqREaEN5EeugoUvD dVzH9JNz5takdCsH6O1qI7HB3OF0cVAqvG/sWn7i6vh7dwEndt0utGaBOIlDrWvDhQ aV0VLmGdRRelQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 672C1C5472E; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:30 +0530 Subject: [PATCH v3 05/29] media: iris: implement iris v4l2 file ops Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-5-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=15155; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=ywx9nYUrAFey2RPmAewqoeSXl/GJ2QCM3IVukYhN9pw=; b=0RaQyOjE2tyxiFAbYHINwSwGDMT2+zoUnJav5KVe7inS6TX5D4f4PR5N0YuCk6jXJX8R+jkcf Pd5I2fjy6laABWznABTJtRnL2btHDyuGPXLvNa0OqQ+w+GG10KIXM6Z X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal Implement open, close and poll ops. Open: Configure the vb2 queue and v4l2 file handler. Allocate a video instance and add the instance to core instance list. Close: Free the instance and remove it from core instance list. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/Kconfig | 1 + drivers/media/platform/qcom/iris/Makefile | 5 +- drivers/media/platform/qcom/iris/iris_core.h | 2 + drivers/media/platform/qcom/iris/iris_hfi_gen1.h | 13 ++ .../platform/qcom/iris/iris_hfi_gen1_command.c | 12 ++ drivers/media/platform/qcom/iris/iris_hfi_gen2.h | 22 +++ .../platform/qcom/iris/iris_hfi_gen2_command.c | 11 ++ drivers/media/platform/qcom/iris/iris_instance.h | 31 ++++ .../platform/qcom/iris/iris_platform_common.h | 1 + .../platform/qcom/iris/iris_platform_sm8250.c | 2 + .../platform/qcom/iris/iris_platform_sm8550.c | 2 + drivers/media/platform/qcom/iris/iris_probe.c | 3 + drivers/media/platform/qcom/iris/iris_vidc.c | 174 +++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_vidc.h | 15 ++ 14 files changed, 293 insertions(+), 1 deletion(-) diff --git a/drivers/media/platform/qcom/iris/Kconfig b/drivers/media/platform/qcom/iris/Kconfig index a9986aaa448d..4b757bb8adf3 100644 --- a/drivers/media/platform/qcom/iris/Kconfig +++ b/drivers/media/platform/qcom/iris/Kconfig @@ -2,6 +2,7 @@ config VIDEO_QCOM_IRIS tristate "Qualcomm Iris V4L2 decoder driver" depends on VIDEO_DEV depends on ARCH_QCOM || COMPILE_TEST + select V4L2_MEM2MEM_DEV help This is a V4L2 driver for Qualcomm Iris video accelerator hardware. It accelerates decoding operations on various diff --git a/drivers/media/platform/qcom/iris/Makefile b/drivers/media/platform/qcom/iris/Makefile index 3e8474d064f4..dcc393a3780a 100644 --- a/drivers/media/platform/qcom/iris/Makefile +++ b/drivers/media/platform/qcom/iris/Makefile @@ -1,6 +1,9 @@ -iris-objs += iris_platform_sm8250.o \ +iris-objs += iris_hfi_gen1_command.o \ + iris_hfi_gen2_command.o \ + iris_platform_sm8250.o \ iris_platform_sm8550.o \ iris_probe.o \ iris_resources.o \ + iris_vidc.o \ obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/iris/iris_core.h b/drivers/media/platform/qcom/iris/iris_core.h index a1afd4387b3a..14489bd4474c 100644 --- a/drivers/media/platform/qcom/iris/iris_core.h +++ b/drivers/media/platform/qcom/iris/iris_core.h @@ -20,6 +20,7 @@ * @irq: iris irq * @v4l2_dev: a holder for v4l2 device structure * @vdev_dec: iris video device structure for decoder + * @iris_v4l2_file_ops: iris v4l2 file ops * @icc_tbl: table of iris interconnects * @icc_count: count of iris interconnects * @pmdomain_tbl: table of iris power domains @@ -36,6 +37,7 @@ struct iris_core { int irq; struct v4l2_device v4l2_dev; struct video_device *vdev_dec; + const struct v4l2_file_operations *iris_v4l2_file_ops; struct icc_bulk_data *icc_tbl; u32 icc_count; struct dev_pm_domain_list *pmdomain_tbl; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1.h new file mode 100644 index 000000000000..b02f629a9cdc --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_HFI_GEN1_H_ +#define _IRIS_HFI_GEN1_H_ + +struct iris_inst; + +struct iris_inst *iris_hfi_gen1_get_instance(void); + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c new file mode 100644 index 000000000000..20c68f4ffb72 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c @@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_hfi_gen1.h" +#include "iris_instance.h" + +struct iris_inst *iris_hfi_gen1_get_instance(void) +{ + return kzalloc(sizeof(struct iris_inst), GFP_KERNEL); +} diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2.h new file mode 100644 index 000000000000..4f9748cbe0e3 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2.h @@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_HFI_GEN2_H_ +#define _IRIS_HFI_GEN2_H_ + +#include "iris_instance.h" + +/** + * struct iris_inst_hfi_gen2 - holds per video instance parameters for hfi_gen2 + * + * @inst: pointer to iris_instance structure + */ +struct iris_inst_hfi_gen2 { + struct iris_inst inst; +}; + +struct iris_inst *iris_hfi_gen2_get_instance(void); + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c new file mode 100644 index 000000000000..3ee33c8befae --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c @@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_hfi_gen2.h" + +struct iris_inst *iris_hfi_gen2_get_instance(void) +{ + return kzalloc(sizeof(struct iris_inst_hfi_gen2), GFP_KERNEL); +} diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h new file mode 100644 index 000000000000..63cb9d70166f --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_instance.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_INSTANCE_H_ +#define _IRIS_INSTANCE_H_ + +#include "iris_core.h" + +/** + * struct iris_inst - holds per video instance parameters + * + * @core: pointer to core structure + * @ctx_q_lock: lock to serialize queues related ioctls + * @lock: lock to seralise forward and reverse threads + * @fh: reference of v4l2 file handler + * @m2m_dev: a reference to m2m device structure + * @m2m_ctx: a reference to m2m context structure + */ + +struct iris_inst { + struct iris_core *core; + struct mutex ctx_q_lock;/* lock to serialize queues related ioctls */ + struct mutex lock; /* lock to serialize forward and reverse threads */ + struct v4l2_fh fh; + struct v4l2_m2m_dev *m2m_dev; + struct v4l2_m2m_ctx *m2m_ctx; +}; + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h index 293fb7e904b0..daa6a74b376f 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_common.h +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h @@ -21,6 +21,7 @@ struct platform_clk_data { }; struct iris_platform_data { + struct iris_inst *(*get_instance)(void); const struct icc_info *icc_tbl; unsigned int icc_tbl_size; const char * const *pmdomain_tbl; diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c index b6f08fc327b5..5d4211a93bc1 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c @@ -6,6 +6,7 @@ #include "iris_core.h" #include "iris_platform_common.h" #include "iris_resources.h" +#include "iris_hfi_gen1.h" static const struct icc_info sm8250_icc_table[] = { { "cpu-cfg", 1000, 1000 }, @@ -25,6 +26,7 @@ static const struct platform_clk_data sm8250_clk_table[] = { }; struct iris_platform_data sm8250_data = { + .get_instance = iris_hfi_gen1_get_instance, .icc_tbl = sm8250_icc_table, .icc_tbl_size = ARRAY_SIZE(sm8250_icc_table), .clk_rst_tbl = sm8250_clk_reset_table, diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c index f2f9e6f6775f..e0a3a162c55f 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c @@ -4,6 +4,7 @@ */ #include "iris_core.h" +#include "iris_hfi_gen2.h" #include "iris_platform_common.h" #include "iris_resources.h" @@ -25,6 +26,7 @@ static const struct platform_clk_data sm8550_clk_table[] = { }; struct iris_platform_data sm8550_data = { + .get_instance = iris_hfi_gen2_get_instance, .icc_tbl = sm8550_icc_table, .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table), .clk_rst_tbl = sm8550_clk_reset_table, diff --git a/drivers/media/platform/qcom/iris/iris_probe.c b/drivers/media/platform/qcom/iris/iris_probe.c index 2616a31224f9..fdf9f12c7728 100644 --- a/drivers/media/platform/qcom/iris/iris_probe.c +++ b/drivers/media/platform/qcom/iris/iris_probe.c @@ -6,6 +6,7 @@ #include #include "iris_core.h" +#include "iris_vidc.h" static int iris_register_video_device(struct iris_core *core) { @@ -18,6 +19,7 @@ static int iris_register_video_device(struct iris_core *core) strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name)); vdev->release = video_device_release; + vdev->fops = core->iris_v4l2_file_ops; vdev->vfl_dir = VFL_DIR_M2M; vdev->v4l2_dev = &core->v4l2_dev; vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING; @@ -76,6 +78,7 @@ static int iris_probe(struct platform_device *pdev) return ret; } + iris_init_ops(core); ret = iris_init_resources(core); if (ret) { dev_err_probe(core->dev, ret, "init resource failed\n"); diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c new file mode 100644 index 000000000000..24a6d744deaf --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -0,0 +1,174 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include + +#include "iris_vidc.h" +#include "iris_instance.h" +#include "iris_platform_common.h" + +#define IRIS_DRV_NAME "iris_driver" +#define IRIS_BUS_NAME "platform:iris_icc" +#define STEP_WIDTH 1 +#define STEP_HEIGHT 1 + +static int iris_v4l2_fh_init(struct iris_inst *inst) +{ + struct iris_core *core = inst->core; + + if (inst->fh.vdev) + return -EINVAL; + + v4l2_fh_init(&inst->fh, core->vdev_dec); + v4l2_fh_add(&inst->fh); + + return 0; +} + +static void iris_v4l2_fh_deinit(struct iris_inst *inst) +{ + if (!inst->fh.vdev) + return; + + v4l2_fh_del(&inst->fh); + inst->fh.ctrl_handler = NULL; + v4l2_fh_exit(&inst->fh); +} + +static struct iris_inst *iris_get_inst(struct file *filp, void *fh) +{ + if (!filp || !filp->private_data) + return NULL; + + return container_of(filp->private_data, + struct iris_inst, fh); +} + +static void iris_m2m_device_run(void *priv) +{ +} + +static void iris_m2m_job_abort(void *priv) +{ + struct iris_inst *inst = priv; + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + + v4l2_m2m_job_finish(inst->m2m_dev, m2m_ctx); +} + +static const struct v4l2_m2m_ops iris_m2m_ops = { + .device_run = iris_m2m_device_run, + .job_abort = iris_m2m_job_abort, +}; + +static int +iris_m2m_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq) +{ + struct iris_inst *inst = priv; + int ret; + + src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; + src_vq->io_modes = VB2_MMAP | VB2_DMABUF; + src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + src_vq->drv_priv = inst; + src_vq->dev = inst->core->dev; + src_vq->lock = &inst->ctx_q_lock; + ret = vb2_queue_init(src_vq); + if (ret) + return ret; + + dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + dst_vq->io_modes = VB2_MMAP | VB2_DMABUF; + dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + dst_vq->drv_priv = inst; + dst_vq->dev = inst->core->dev; + dst_vq->lock = &inst->ctx_q_lock; + + return vb2_queue_init(dst_vq); +} + +int iris_open(struct file *filp) +{ + struct iris_core *core = video_drvdata(filp); + struct iris_inst *inst = NULL; + int ret; + + inst = core->iris_platform_data->get_instance(); + if (!inst) + return -ENOMEM; + + inst->core = core; + + mutex_init(&inst->lock); + mutex_init(&inst->ctx_q_lock); + + ret = iris_v4l2_fh_init(inst); + if (ret) + goto fail_free_inst; + + inst->m2m_dev = v4l2_m2m_init(&iris_m2m_ops); + if (IS_ERR_OR_NULL(inst->m2m_dev)) { + ret = -EINVAL; + goto fail_v4l2_fh_deinit; + } + + inst->m2m_ctx = v4l2_m2m_ctx_init(inst->m2m_dev, inst, iris_m2m_queue_init); + if (IS_ERR_OR_NULL(inst->m2m_ctx)) { + ret = -EINVAL; + goto fail_m2m_release; + } + + inst->fh.m2m_ctx = inst->m2m_ctx; + filp->private_data = &inst->fh; + + return 0; + +fail_m2m_release: + v4l2_m2m_release(inst->m2m_dev); +fail_v4l2_fh_deinit: + iris_v4l2_fh_deinit(inst); +fail_free_inst: + mutex_destroy(&inst->ctx_q_lock); + mutex_destroy(&inst->lock); + kfree(inst); + + return ret; +} + +int iris_close(struct file *filp) +{ + struct iris_inst *inst; + + inst = iris_get_inst(filp, NULL); + if (!inst) + return -EINVAL; + + v4l2_m2m_ctx_release(inst->m2m_ctx); + v4l2_m2m_release(inst->m2m_dev); + mutex_lock(&inst->lock); + iris_v4l2_fh_deinit(inst); + mutex_unlock(&inst->lock); + mutex_destroy(&inst->ctx_q_lock); + mutex_destroy(&inst->lock); + kfree(inst); + filp->private_data = NULL; + + return 0; +} + +static struct v4l2_file_operations iris_v4l2_file_ops = { + .owner = THIS_MODULE, + .open = iris_open, + .release = iris_close, + .unlocked_ioctl = video_ioctl2, + .poll = v4l2_m2m_fop_poll, + .mmap = v4l2_m2m_fop_mmap, +}; + +void iris_init_ops(struct iris_core *core) +{ + core->iris_v4l2_file_ops = &iris_v4l2_file_ops; +} diff --git a/drivers/media/platform/qcom/iris/iris_vidc.h b/drivers/media/platform/qcom/iris/iris_vidc.h new file mode 100644 index 000000000000..f09862e6199c --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_vidc.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_VIDC_H_ +#define _IRIS_VIDC_H_ + +struct iris_core; + +void iris_init_ops(struct iris_core *core); +int iris_open(struct file *filp); +int iris_close(struct file *filp); + +#endif From patchwork Tue Aug 27 10:05:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 824193 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E772619DF8D; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; cv=none; b=Tnn+dtdJb8MuHVumXLrbDnIBINu2iZNiS3U0qsVrkahaOy/9RzQjh7rMYnlUFm844uFVy9hdXiRaZNAeGCfhpCpyGM1HMAPa7AdCM2tNqn/oR+TuOWoe5SYfhwggL55rqUpH7lfonACpdlCEJT1Atu+nSvSZKhqg5uvaD16Or+8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; c=relaxed/simple; bh=OB5Rp6gsgfos2TFAJYV37NyV+/pYeOFg27UfU7DNr5Q=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=eakGtUKOEYGJxjtJIAgij3UWogL3ykqrVG0ub3wYroy7i5Ykg95BYd+buZ4sm8F8SL9FAv5rR7pt+7Af8Dg85Lu/3XudZSjW9nV4Irv5ycF01bx1LrSxi5f5FUK7nZHHuUJ3VLfhcFGYB5qAnRisYybvmYxXjxn6/Cl7LIxiwzI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lWi1YW8m; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lWi1YW8m" Received: by smtp.kernel.org (Postfix) with ESMTPS id 8ADC2C4AF52; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753194; bh=OB5Rp6gsgfos2TFAJYV37NyV+/pYeOFg27UfU7DNr5Q=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=lWi1YW8mMEjuMLHwzlpBU9hUSv16q4Vmg1FkKv/jAfOZJ5I6EnFHlrxB3SP9e1taX n0Fyo3HeEihwipw8tVNmb7ZCJYFGrvXtvt/R6m4chmf9E4vWZdyhHZGVHouVfF/MhT S6uzjSSSL765msHGILrXTKqtgT/A+/7aliEW0i2I5d4KFBkylyWvRzIS/1m7hy1NOP 9gKCQ/yU6yCKk74IGUdxtP+3ipLwFc91oBD2+1NZfj7m80xqxw4yH8pmPs3Ux5r4Iz q49zAy7zzweTKPNsXk7TfpVb6RPbX1SGIvpd6wacuZAZbw38aujseSf5jg11DUDjJP YUxJvoqyzPlSQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 81CBDC52D6F; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:31 +0530 Subject: [PATCH v3 06/29] media: iris: introduce iris core state management with shared queues Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-6-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=21630; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=LxcvU47s32KaX6LlNTKQBKpSaR+rju6FxEtyCqXceNg=; b=Q2ZXW0Xlc3tmvUgqNitIP4VQj74k5MHoRG489mOy/3wNqR/NMyhFioFQX8vu5lOJewCNXOkYb qYyvjV1nhCNC1lzYqUklh+nwaW4C48mLnmDH1NBlslEoNtohY7uSGXN X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal Introduce core state management for iris driver with necessary queues needed for host firmware communication. There are 3 types of queues: Command queue - driver to write any command to firmware. Message queue - firmware to send any response to driver. Debug queue - firmware to write debug messages. Initialize and configire shared queues during probe. Different states for core: IRIS_CORE_DEINIT - default state. IRIS_CORE_INIT - core state with core initialized. FW loaded and HW brought out of reset, shared queues established between host driver and firmware. IRIS_CORE_ERROR - error state. ----------- | V ----------- | DEINIT | ----------- ^ / \ / \ / \ / \ v v ----------- ----------. | INIT |-->| ERROR | ----------- ----------. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/Makefile | 4 +- drivers/media/platform/qcom/iris/iris_core.c | 46 ++++++ drivers/media/platform/qcom/iris/iris_core.h | 23 +++ drivers/media/platform/qcom/iris/iris_hfi_queue.c | 126 +++++++++++++++ drivers/media/platform/qcom/iris/iris_hfi_queue.h | 175 +++++++++++++++++++++ .../platform/qcom/iris/iris_platform_common.h | 1 + .../platform/qcom/iris/iris_platform_sm8250.c | 1 + .../platform/qcom/iris/iris_platform_sm8550.c | 1 + drivers/media/platform/qcom/iris/iris_probe.c | 20 +++ drivers/media/platform/qcom/iris/iris_state.h | 41 +++++ drivers/media/platform/qcom/iris/iris_vidc.c | 6 + 11 files changed, 443 insertions(+), 1 deletion(-) diff --git a/drivers/media/platform/qcom/iris/Makefile b/drivers/media/platform/qcom/iris/Makefile index dcc393a3780a..48d54e53a576 100644 --- a/drivers/media/platform/qcom/iris/Makefile +++ b/drivers/media/platform/qcom/iris/Makefile @@ -1,5 +1,7 @@ -iris-objs += iris_hfi_gen1_command.o \ +iris-objs += iris_core.o \ + iris_hfi_gen1_command.o \ iris_hfi_gen2_command.o \ + iris_hfi_queue.o \ iris_platform_sm8250.o \ iris_platform_sm8550.o \ iris_probe.o \ diff --git a/drivers/media/platform/qcom/iris/iris_core.c b/drivers/media/platform/qcom/iris/iris_core.c new file mode 100644 index 000000000000..360a54909ef6 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_core.c @@ -0,0 +1,46 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_core.h" +#include "iris_state.h" + +void iris_core_deinit(struct iris_core *core) +{ + mutex_lock(&core->lock); + iris_hfi_queues_deinit(core); + core->state = IRIS_CORE_DEINIT; + mutex_unlock(&core->lock); +} + +int iris_core_init(struct iris_core *core) +{ + int ret; + + mutex_lock(&core->lock); + if (core->state == IRIS_CORE_INIT) { + ret = 0; + goto exit; + } else if (core->state == IRIS_CORE_ERROR) { + ret = -EINVAL; + goto error; + } + + core->state = IRIS_CORE_INIT; + + ret = iris_hfi_queues_init(core); + if (ret) + goto error; + + mutex_unlock(&core->lock); + + return 0; + +error: + core->state = IRIS_CORE_DEINIT; +exit: + mutex_unlock(&core->lock); + + return ret; +} diff --git a/drivers/media/platform/qcom/iris/iris_core.h b/drivers/media/platform/qcom/iris/iris_core.h index 14489bd4474c..13c5932f9110 100644 --- a/drivers/media/platform/qcom/iris/iris_core.h +++ b/drivers/media/platform/qcom/iris/iris_core.h @@ -9,8 +9,10 @@ #include #include +#include "iris_hfi_queue.h" #include "iris_platform_common.h" #include "iris_resources.h" +#include "iris_state.h" /** * struct iris_core - holds core parameters valid for all instances @@ -29,6 +31,15 @@ * @clk_count: count of iris clocks * @resets: table of iris reset clocks * @iris_platform_data: a structure for platform data + * @state: current state of core + * @iface_q_table_daddr: device address for interface queue table memory + * @sfr_daddr: device address for SFR (Sub System Failure Reason) register memory + * @iface_q_table_vaddr: virtual address for interface queue table memory + * @sfr_vaddr: virtual address for SFR (Sub System Failure Reason) register memory + * @command_queue: shared interface queue to send commands to firmware + * @message_queue: shared interface queue to receive responses from firmware + * @debug_queue: shared interface queue to receive debug info from firmware + * @lock: a lock for this strucure */ struct iris_core { @@ -46,6 +57,18 @@ struct iris_core { u32 clk_count; struct reset_control_bulk_data *resets; const struct iris_platform_data *iris_platform_data; + enum iris_core_state state; + dma_addr_t iface_q_table_daddr; + dma_addr_t sfr_daddr; + void *iface_q_table_vaddr; + void *sfr_vaddr; + struct iris_iface_q_info command_queue; + struct iris_iface_q_info message_queue; + struct iris_iface_q_info debug_queue; + struct mutex lock; /* lock for core related operations */ }; +int iris_core_init(struct iris_core *core); +void iris_core_deinit(struct iris_core *core); + #endif diff --git a/drivers/media/platform/qcom/iris/iris_hfi_queue.c b/drivers/media/platform/qcom/iris/iris_hfi_queue.c new file mode 100644 index 000000000000..11938880b8cd --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_hfi_queue.c @@ -0,0 +1,126 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_core.h" +#include "iris_hfi_queue.h" + +static void iris_hfi_queue_set_header(struct iris_core *core, u32 queue_id, + struct iris_iface_q_info *iface_q) +{ + iface_q->qhdr->status = 0x1; + iface_q->qhdr->start_addr = iface_q->device_addr; + iface_q->qhdr->header_type = IFACEQ_DFLT_QHDR; + iface_q->qhdr->queue_type = queue_id; + iface_q->qhdr->q_size = IFACEQ_QUEUE_SIZE / sizeof(u32); + iface_q->qhdr->pkt_size = 0; /* variable packet size */ + iface_q->qhdr->rx_wm = 0x1; + iface_q->qhdr->tx_wm = 0x1; + iface_q->qhdr->rx_req = 0x1; + iface_q->qhdr->tx_req = 0x0; + iface_q->qhdr->rx_irq_status = 0x0; + iface_q->qhdr->tx_irq_status = 0x0; + iface_q->qhdr->read_idx = 0x0; + iface_q->qhdr->write_idx = 0x0; + + /* + * Set receive request to zero on debug queue as there is no + * need of interrupt from video hardware for debug messages + */ + if (queue_id == IFACEQ_DBGQ_ID) + iface_q->qhdr->rx_req = 0; +} + +static void +iris_hfi_queue_init(struct iris_core *core, u32 queue_id, struct iris_iface_q_info *iface_q) +{ + struct iris_hfi_queue_table_header *q_tbl_hdr; + unsigned int offset = 0; + + q_tbl_hdr = core->iface_q_table_vaddr; + + offset = sizeof(*q_tbl_hdr) + (queue_id * IFACEQ_QUEUE_SIZE); + iface_q->device_addr = core->iface_q_table_daddr + offset; + iface_q->kernel_vaddr = + (void *)((char *)core->iface_q_table_vaddr + offset); + iface_q->qhdr = &q_tbl_hdr->q_hdr[queue_id]; + + iris_hfi_queue_set_header(core, queue_id, iface_q); +} + +static void iris_hfi_queue_deinit(struct iris_iface_q_info *iface_q) +{ + iface_q->qhdr = NULL; + iface_q->kernel_vaddr = NULL; + iface_q->device_addr = 0; +} + +int iris_hfi_queues_init(struct iris_core *core) +{ + struct iris_hfi_queue_table_header *q_tbl_hdr; + u32 queue_size; + + /* Iris hardware requires 4K queue alignment */ + queue_size = ALIGN((sizeof(*q_tbl_hdr) + (IFACEQ_QUEUE_SIZE * IFACEQ_NUMQ)), SZ_4K); + core->iface_q_table_vaddr = dma_alloc_attrs(core->dev, queue_size, + &core->iface_q_table_daddr, + GFP_KERNEL, DMA_ATTR_WRITE_COMBINE); + if (!core->iface_q_table_vaddr) { + dev_err(core->dev, "queues alloc and map failed\n"); + return -ENOMEM; + } + + core->sfr_vaddr = dma_alloc_attrs(core->dev, SFR_SIZE, + &core->sfr_daddr, + GFP_KERNEL, DMA_ATTR_WRITE_COMBINE); + if (!core->sfr_vaddr) { + dev_err(core->dev, "sfr alloc and map failed\n"); + dma_free_attrs(core->dev, sizeof(*q_tbl_hdr), core->iface_q_table_vaddr, + core->iface_q_table_daddr, DMA_ATTR_WRITE_COMBINE); + return -ENOMEM; + } + + iris_hfi_queue_init(core, IFACEQ_CMDQ_ID, &core->command_queue); + iris_hfi_queue_init(core, IFACEQ_MSGQ_ID, &core->message_queue); + iris_hfi_queue_init(core, IFACEQ_DBGQ_ID, &core->debug_queue); + + q_tbl_hdr = (struct iris_hfi_queue_table_header *)core->iface_q_table_vaddr; + q_tbl_hdr->version = 0; + q_tbl_hdr->device_addr = (void *)core; + strscpy(q_tbl_hdr->name, "iris-hfi-queues", sizeof(q_tbl_hdr->name)); + q_tbl_hdr->size = sizeof(*q_tbl_hdr); + q_tbl_hdr->qhdr0_offset = sizeof(*q_tbl_hdr) - + (IFACEQ_NUMQ * sizeof(struct iris_hfi_queue_header)); + q_tbl_hdr->qhdr_size = sizeof(q_tbl_hdr->q_hdr[0]); + q_tbl_hdr->num_q = IFACEQ_NUMQ; + q_tbl_hdr->num_active_q = IFACEQ_NUMQ; + + /* Write sfr size in first word to be used by firmware */ + *((u32 *)core->sfr_vaddr) = SFR_SIZE; + + return 0; +} + +void iris_hfi_queues_deinit(struct iris_core *core) +{ + if (!core->iface_q_table_vaddr) + return; + + iris_hfi_queue_deinit(&core->debug_queue); + iris_hfi_queue_deinit(&core->message_queue); + iris_hfi_queue_deinit(&core->command_queue); + + dma_free_attrs(core->dev, SFR_SIZE, core->sfr_vaddr, + core->sfr_daddr, DMA_ATTR_WRITE_COMBINE); + + core->sfr_vaddr = NULL; + core->sfr_daddr = 0; + + dma_free_attrs(core->dev, sizeof(struct iris_hfi_queue_table_header), + core->iface_q_table_vaddr, core->iface_q_table_daddr, + DMA_ATTR_WRITE_COMBINE); + + core->iface_q_table_vaddr = NULL; + core->iface_q_table_daddr = 0; +} diff --git a/drivers/media/platform/qcom/iris/iris_hfi_queue.h b/drivers/media/platform/qcom/iris/iris_hfi_queue.h new file mode 100644 index 000000000000..54994bb776f1 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_hfi_queue.h @@ -0,0 +1,175 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_HFI_QUEUE_H_ +#define _IRIS_HFI_QUEUE_H_ + +struct iris_core; + +/* + * Maximum number of buffers which queue can hold until + * hardware stops responding and driver times out. + */ +#define IFACEQ_MAX_BUF_COUNT 50 +/* + * Max session supported are 16. + * this value is used to calcualte the size of + * individual shared queue. + */ +#define IFACE_MAX_PARALLEL_SESSIONS 16 +#define IFACEQ_DFLT_QHDR 0x0101 +#define IFACEQ_MAX_PKT_SIZE 1024 /* Maximum size of a packet in the queue */ + +/* + * SFR: Subsystem Failure Reason + * when hardware goes into bad state/failure, firmware fills this memory + * and driver will get to know the actual failure reason from this SFR buffer. + */ +#define SFR_SIZE SZ_4K /* Iris hardware requires 4K queue alignment */ + +#define IFACEQ_QUEUE_SIZE (IFACEQ_MAX_PKT_SIZE * \ + IFACEQ_MAX_BUF_COUNT * IFACE_MAX_PARALLEL_SESSIONS) + +/* + * Memory layout of the shared queues: + * + * ||=================|| ^ ^ ^ + * || || | | | + * || Queue Table || 288 Bytes | | + * || Header || | | | + * || || | | | + * ||-----------------|| V | | + * ||-----------------|| ^ | | + * || || | | | + * || Command Queue || 56 Bytes | | + * || Header || | | | + * || || | | | + * ||-----------------|| V 456 Bytes | + * ||-----------------|| ^ | | + * || || | | | + * || Message Queue || 56 Bytes | | + * || Header || | | | + * || || | | | + * ||-----------------|| V | Buffer size aligned to 4k + * ||-----------------|| ^ | Overall Queue Size = 2,404 KB + * || || | | | + * || Debug Queue || 56 Bytes | | + * || Header || | | | + * || || | | | + * ||=================|| V V | + * ||=================|| ^ | + * || || | | + * || Command || 800 KB | + * || Queue || | | + * || || | | + * ||=================|| V | + * ||=================|| ^ | + * || || | | + * || Message || 800 KB | + * || Queue || | | + * || || | | + * ||=================|| V | + * ||=================|| ^ | + * || || | | + * || Debug || 800 KB | + * || Queue || | | + * || || | | + * ||=================|| V | + * || || | + * ||=================|| V + */ + +/* + * Shared queues are used for communication between driver and firmware. + * There are 3 types of queues: + * Command queue - driver to write any command to firmware. + * Message queue - firmware to send any response to driver. + * Debug queue - firmware to write debug message. + */ + +/* Host-firmware shared queue ids */ +enum iris_iface_queue { + IFACEQ_CMDQ_ID, + IFACEQ_MSGQ_ID, + IFACEQ_DBGQ_ID, + IFACEQ_NUMQ, /* not an index */ +}; + +/** + * struct iris_hfi_queue_header + * + * @status: Queue status, qhdr_state define possible status + * @start_addr: Queue start address in non cached memory + * @type: qhdr_tx, qhdr_rx, qhdr_q_id and priority defines qhdr type + * @q_size: Queue size + * Number of queue packets if pkt_size is non-zero + * Queue size in bytes if pkt_size is zero + * @pkt_size: Size of queue packet entries + * 0x0: variable queue packet size + * non zero: size of queue packet entry, fixed + * @pkt_drop_cnt: Number of packets dropped by sender + * @rx_wm: Receiver watermark, applicable in event driven mode + * @tx_wm: Sender watermark, applicable in event driven mode + * @rx_req: Receiver sets this bit if queue is empty + * @tx_req: Sender sets this bit if queue is full + * @rx_irq_status: Receiver sets this bit and triggers an interrupt to + * the sender after packets are dequeued. Sender clears this bit + * @tx_irq_status: Sender sets this bit and triggers an interrupt to + * the receiver after packets are queued. Receiver clears this bit + * @read_idx: Index till where receiver has consumed the packets from the queue. + * @write_idx: Index till where sender has written the packets into the queue. + */ +struct iris_hfi_queue_header { + u32 status; + u32 start_addr; + u16 queue_type; + u16 header_type; + u32 q_size; + u32 pkt_size; + u32 pkt_drop_cnt; + u32 rx_wm; + u32 tx_wm; + u32 rx_req; + u32 tx_req; + u32 rx_irq_status; + u32 tx_irq_status; + u32 read_idx; + u32 write_idx; +}; + +/** + * struct iris_hfi_queue_table_header + * + * @version: Queue table version number + * @size: Queue table size from version to last parametr in qhdr entry + * @qhdr0_offset: Offset to the start of first qhdr + * @qhdr_size: Queue header size in bytes + * @num_q: Total number of queues in Queue table + * @num_active_q: Total number of active queues + * @device_addr: Device address of the queue + * @name: Queue name in characters + */ +struct iris_hfi_queue_table_header { + u32 version; + u32 size; + u32 qhdr0_offset; + u32 qhdr_size; + u32 num_q; + u32 num_active_q; + void *device_addr; + char name[256]; /* NUL-terminated array of characters */ + struct iris_hfi_queue_header q_hdr[IFACEQ_NUMQ]; +}; + +struct iris_iface_q_info { + struct iris_hfi_queue_header *qhdr; + dma_addr_t device_addr; + void *kernel_vaddr; +}; + +int iris_hfi_queues_init(struct iris_core *core); +void iris_hfi_queues_deinit(struct iris_core *core); + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h index daa6a74b376f..5287fdf913bc 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_common.h +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h @@ -32,6 +32,7 @@ struct iris_platform_data { unsigned int clk_tbl_size; const char * const *clk_rst_tbl; unsigned int clk_rst_tbl_size; + u64 dma_mask; }; #endif diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c index 5d4211a93bc1..664cb2b359a3 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c @@ -37,4 +37,5 @@ struct iris_platform_data sm8250_data = { .opp_pd_tbl_size = ARRAY_SIZE(sm8250_opp_pd_table), .clk_tbl = sm8250_clk_table, .clk_tbl_size = ARRAY_SIZE(sm8250_clk_table), + .dma_mask = GENMASK(31, 29) - 1, }; diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c index e0a3a162c55f..0db832b099ac 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c @@ -37,4 +37,5 @@ struct iris_platform_data sm8550_data = { .opp_pd_tbl_size = ARRAY_SIZE(sm8550_opp_pd_table), .clk_tbl = sm8550_clk_table, .clk_tbl_size = ARRAY_SIZE(sm8550_clk_table), + .dma_mask = GENMASK(31, 29) - 1, }; diff --git a/drivers/media/platform/qcom/iris/iris_probe.c b/drivers/media/platform/qcom/iris/iris_probe.c index fdf9f12c7728..64bac287d3a8 100644 --- a/drivers/media/platform/qcom/iris/iris_probe.c +++ b/drivers/media/platform/qcom/iris/iris_probe.c @@ -47,15 +47,21 @@ static void iris_remove(struct platform_device *pdev) if (!core) return; + iris_core_deinit(core); + video_unregister_device(core->vdev_dec); v4l2_device_unregister(&core->v4l2_dev); + + mutex_destroy(&core->lock); + core->state = IRIS_CORE_DEINIT; } static int iris_probe(struct platform_device *pdev) { struct device *dev = &pdev->dev; struct iris_core *core; + u64 dma_mask; int ret; core = devm_kzalloc(&pdev->dev, sizeof(*core), GFP_KERNEL); @@ -63,6 +69,9 @@ static int iris_probe(struct platform_device *pdev) return -ENOMEM; core->dev = dev; + core->state = IRIS_CORE_DEINIT; + mutex_init(&core->lock); + core->reg_base = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(core->reg_base)) return PTR_ERR(core->reg_base); @@ -95,8 +104,19 @@ static int iris_probe(struct platform_device *pdev) platform_set_drvdata(pdev, core); + dma_mask = core->iris_platform_data->dma_mask; + + ret = dma_set_mask_and_coherent(dev, dma_mask); + if (ret) + goto err_vdev_unreg; + + dma_set_max_seg_size(&pdev->dev, DMA_BIT_MASK(32)); + dma_set_seg_boundary(&pdev->dev, DMA_BIT_MASK(64)); + return 0; +err_vdev_unreg: + video_unregister_device(core->vdev_dec); err_v4l2_unreg: v4l2_device_unregister(&core->v4l2_dev); diff --git a/drivers/media/platform/qcom/iris/iris_state.h b/drivers/media/platform/qcom/iris/iris_state.h new file mode 100644 index 000000000000..a0bfee98578a --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_state.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_STATE_H_ +#define _IRIS_STATE_H_ + +/** + * enum iris_core_state + * + * IRIS_CORE_DEINIT: default state. + * IRIS_CORE_INIT: core state with core initialized. FW loaded and + * HW brought out of reset, shared queues established + * between host driver and firmware. + * IRIS_CORE_ERROR: error state. + * + * ----------- + * | + * V + * ----------- + * +--->| DEINIT |<---+ + * | ----------- | + * | | | + * | v | + * | ----------- | + * | / \ | + * | / \ | + * | / \ | + * | v v v + * ----------- ----------- + * | INIT |--->| ERROR | + * ----------- ----------- + */ +enum iris_core_state { + IRIS_CORE_DEINIT, + IRIS_CORE_INIT, + IRIS_CORE_ERROR, +}; + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index 24a6d744deaf..a5c4e73ae531 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -96,6 +96,12 @@ int iris_open(struct file *filp) struct iris_inst *inst = NULL; int ret; + ret = iris_core_init(core); + if (ret) { + dev_err(core->dev, "core init failed\n"); + return ret; + } + inst = core->iris_platform_data->get_instance(); if (!inst) return -ENOMEM; From patchwork Tue Aug 27 10:05:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 822727 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E9D619E7DF; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; cv=none; b=JRcWfOgOfOCDHRVx1ffbJGB7EEcj062nyb5Orf4dj1Sw0Ox/IFTVEsxHMTHS3anzEoJkvT1Xypk1ehkCvBiYycK5szahrwenOBdJxbeHCeiEjqQJ3Dn51XCkU7A0+HD3vpE5z7vC8xTSWLkz+cRcJXX822LVR6cgljXKCSWSGbg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; c=relaxed/simple; bh=GwIO+Ge/Pd4kA/s7XrF8gZmyUOGZvnQGbYrU6qhTL8E=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=AIsMCDV1vC+ZaVypNkPKVaqLL2lSO4UokUbNLDiiqr2lrra+LbwKbfyDD41AbMhYNdEafajjrTCvkyysgQCJc1YQAGn9MmXEDBvVsmm0F0m+vk1lgbKqhJRwfV3+DUw4PsISEoSyhqwph4EVF7bSKK7tuyvhtsvqg+Qp4qc8Nqw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=mQqSd8BK; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="mQqSd8BK" Received: by smtp.kernel.org (Postfix) with ESMTPS id B1A77C4AF63; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753194; bh=GwIO+Ge/Pd4kA/s7XrF8gZmyUOGZvnQGbYrU6qhTL8E=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=mQqSd8BKwlAvHuKpLTP4jiJ57adYRZAEgohmCaskrSc33XeS9ANVjguvYsP/6r/me oyRxCo955LDFr1pMJygGiniP/OkBtJaKyAZw+LJou5VJb3laVhOoJVT8UCv0+BPQ5y /Gw1ozjhJL5iVaakahbj/n/qW0Y5TmHIsGuJzHuLmLtX/9xNmIgMjK0j87eXynA6an O7V3QGuV9PxY6fhKydm9SBbfnDrWtF1dGIBNU8ZE0Z5M349SnPA2CDL7vePi/mCaF9 ft0NIf/Dhb2VW3TO9siWHlzKx3cALW5xv2HLTBuc6iTTJo3zRq9tIl4Uk8cCjygUDr O9DMXQvFV9lqQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id A255DC5472E; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:32 +0530 Subject: [PATCH v3 07/29] media: iris: implement video firmware load/unload Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-7-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=10200; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=bQGBq5FPPJkFFZUFXpindbNsVWATyZ0PV6VZQSW2mXE=; b=W6MmSIJ+I2MP32j9yDxygBBVUeD1bZmyNlvMeh8ucXLjgIOrlOLgX4RhO6zKJrILmN/8QahIR AEb52og5TsEAucYUarnUZCcj8FrkdRsxRBK668PJXy+uhd6jMUUvjtB X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal Load/unload firmware in memory via mdt loader. Firmware is loaded as part of core initialization and unloaded as part of core de-initialization. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/Kconfig | 2 + drivers/media/platform/qcom/iris/Makefile | 1 + drivers/media/platform/qcom/iris/iris_core.c | 8 ++ drivers/media/platform/qcom/iris/iris_firmware.c | 146 +++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_firmware.h | 14 ++ .../platform/qcom/iris/iris_platform_common.h | 12 ++ .../platform/qcom/iris/iris_platform_sm8250.c | 10 ++ .../platform/qcom/iris/iris_platform_sm8550.c | 10 ++ 8 files changed, 203 insertions(+) diff --git a/drivers/media/platform/qcom/iris/Kconfig b/drivers/media/platform/qcom/iris/Kconfig index 4b757bb8adf3..ea518760c65c 100644 --- a/drivers/media/platform/qcom/iris/Kconfig +++ b/drivers/media/platform/qcom/iris/Kconfig @@ -3,6 +3,8 @@ config VIDEO_QCOM_IRIS depends on VIDEO_DEV depends on ARCH_QCOM || COMPILE_TEST select V4L2_MEM2MEM_DEV + select QCOM_MDT_LOADER if ARCH_QCOM + select QCOM_SCM help This is a V4L2 driver for Qualcomm Iris video accelerator hardware. It accelerates decoding operations on various diff --git a/drivers/media/platform/qcom/iris/Makefile b/drivers/media/platform/qcom/iris/Makefile index 48d54e53a576..ddd4c994a0b9 100644 --- a/drivers/media/platform/qcom/iris/Makefile +++ b/drivers/media/platform/qcom/iris/Makefile @@ -1,4 +1,5 @@ iris-objs += iris_core.o \ + iris_firmware.o \ iris_hfi_gen1_command.o \ iris_hfi_gen2_command.o \ iris_hfi_queue.o \ diff --git a/drivers/media/platform/qcom/iris/iris_core.c b/drivers/media/platform/qcom/iris/iris_core.c index 360a54909ef6..8c7d53c57086 100644 --- a/drivers/media/platform/qcom/iris/iris_core.c +++ b/drivers/media/platform/qcom/iris/iris_core.c @@ -4,11 +4,13 @@ */ #include "iris_core.h" +#include "iris_firmware.h" #include "iris_state.h" void iris_core_deinit(struct iris_core *core) { mutex_lock(&core->lock); + iris_fw_unload(core); iris_hfi_queues_deinit(core); core->state = IRIS_CORE_DEINIT; mutex_unlock(&core->lock); @@ -33,10 +35,16 @@ int iris_core_init(struct iris_core *core) if (ret) goto error; + ret = iris_fw_load(core); + if (ret) + goto error_queue_deinit; + mutex_unlock(&core->lock); return 0; +error_queue_deinit: + iris_hfi_queues_deinit(core); error: core->state = IRIS_CORE_DEINIT; exit: diff --git a/drivers/media/platform/qcom/iris/iris_firmware.c b/drivers/media/platform/qcom/iris/iris_firmware.c new file mode 100644 index 000000000000..55bbcc798f4c --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_firmware.c @@ -0,0 +1,146 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include +#include +#include +#include +#include + +#include "iris_core.h" +#include "iris_firmware.h" + +#define MAX_FIRMWARE_NAME_SIZE 128 + +static int iris_protect_cp_mem(struct iris_core *core) +{ + struct tz_cp_config *cp_config; + int ret; + + cp_config = core->iris_platform_data->tz_cp_config_data; + + ret = qcom_scm_mem_protect_video_var(cp_config->cp_start, + cp_config->cp_size, + cp_config->cp_nonpixel_start, + cp_config->cp_nonpixel_size); + if (ret) + dev_err(core->dev, "failed to protect memory(%d)\n", ret); + + return ret; +} + +static int iris_load_fw_to_memory(struct iris_core *core, const char *fw_name) +{ + const struct firmware *firmware = NULL; + struct device_node *node = NULL; + struct reserved_mem *rmem; + phys_addr_t mem_phys = 0; + void *mem_virt = NULL; + size_t res_size = 0; + ssize_t fw_size = 0; + struct device *dev; + int pas_id = 0; + int ret; + + if (!fw_name || !(*fw_name) || !core) + return -EINVAL; + + dev = core->dev; + + if (strlen(fw_name) >= MAX_FIRMWARE_NAME_SIZE - 4) + return -EINVAL; + + pas_id = core->iris_platform_data->pas_id; + + node = of_parse_phandle(dev->of_node, "memory-region", 0); + if (!node) + return -EINVAL; + + rmem = of_reserved_mem_lookup(node); + if (!rmem) { + ret = -EINVAL; + goto err_put_node; + } + + mem_phys = rmem->base; + res_size = rmem->size; + + ret = request_firmware(&firmware, fw_name, dev); + if (ret) { + dev_err(core->dev, "failed to request fw \"%s\", error %d\n", + fw_name, ret); + goto err_put_node; + } + + fw_size = qcom_mdt_get_size(firmware); + if (fw_size < 0 || res_size < (size_t)fw_size) { + ret = -EINVAL; + dev_err(core->dev, "out of bound fw image fw size: %ld, res_size: %lu\n", + fw_size, res_size); + goto err_release_fw; + } + + mem_virt = memremap(mem_phys, res_size, MEMREMAP_WC); + if (!mem_virt) { + dev_err(core->dev, "failed to remap fw memory phys %pa[p]\n", + &mem_phys); + goto err_release_fw; + } + + ret = qcom_mdt_load(dev, firmware, fw_name, + pas_id, mem_virt, mem_phys, res_size, NULL); + if (ret) { + dev_err(core->dev, "error %d loading fw \"%s\"\n", + ret, fw_name); + goto err_mem_unmap; + } + ret = qcom_scm_pas_auth_and_reset(pas_id); + if (ret) { + dev_err(core->dev, "error %d authenticating fw \"%s\"\n", + ret, fw_name); + goto err_mem_unmap; + } + + return ret; + +err_mem_unmap: + memunmap(mem_virt); +err_release_fw: + release_firmware(firmware); +err_put_node: + of_node_put(node); + return ret; +} + +int iris_fw_load(struct iris_core *core) +{ + int ret; + + ret = iris_load_fw_to_memory(core, core->iris_platform_data->fwname); + if (ret) { + dev_err(core->dev, "firmware download failed\n"); + return -ENOMEM; + } + + ret = iris_protect_cp_mem(core); + if (ret) { + dev_err(core->dev, "protect memory failed\n"); + qcom_scm_pas_shutdown(core->iris_platform_data->pas_id); + return ret; + } + + return ret; +} + +int iris_fw_unload(struct iris_core *core) +{ + int ret; + + ret = qcom_scm_pas_shutdown(core->iris_platform_data->pas_id); + if (ret) + dev_err(core->dev, "firmware unload failed with ret %d\n", ret); + + return ret; +} diff --git a/drivers/media/platform/qcom/iris/iris_firmware.h b/drivers/media/platform/qcom/iris/iris_firmware.h new file mode 100644 index 000000000000..8d4f6b7f75c5 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_firmware.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_FIRMWARE_H_ +#define _IRIS_FIRMWARE_H_ + +struct iris_core; + +int iris_fw_load(struct iris_core *core); +int iris_fw_unload(struct iris_core *core); + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h index 5287fdf913bc..9c919367f9d7 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_common.h +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h @@ -6,6 +6,8 @@ #ifndef _IRIS_PLATFORM_COMMON_H_ #define _IRIS_PLATFORM_COMMON_H_ +#define IRIS_PAS_ID 9 + extern struct iris_platform_data sm8550_data; extern struct iris_platform_data sm8250_data; @@ -20,6 +22,13 @@ struct platform_clk_data { const char *clk_name; }; +struct tz_cp_config { + u32 cp_start; + u32 cp_size; + u32 cp_nonpixel_start; + u32 cp_nonpixel_size; +}; + struct iris_platform_data { struct iris_inst *(*get_instance)(void); const struct icc_info *icc_tbl; @@ -33,6 +42,9 @@ struct iris_platform_data { const char * const *clk_rst_tbl; unsigned int clk_rst_tbl_size; u64 dma_mask; + const char *fwname; + u32 pas_id; + struct tz_cp_config *tz_cp_config_data; }; #endif diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c index 664cb2b359a3..36bcb1b851ed 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c @@ -25,6 +25,13 @@ static const struct platform_clk_data sm8250_clk_table[] = { {IRIS_HW_CLK, "vcodec0_core" }, }; +static struct tz_cp_config tz_cp_config_sm8250 = { + .cp_start = 0, + .cp_size = 0x25800000, + .cp_nonpixel_start = 0x01000000, + .cp_nonpixel_size = 0x24800000, +}; + struct iris_platform_data sm8250_data = { .get_instance = iris_hfi_gen1_get_instance, .icc_tbl = sm8250_icc_table, @@ -38,4 +45,7 @@ struct iris_platform_data sm8250_data = { .clk_tbl = sm8250_clk_table, .clk_tbl_size = ARRAY_SIZE(sm8250_clk_table), .dma_mask = GENMASK(31, 29) - 1, + .fwname = "qcom/vpu/vpu20_p4.mbn", + .pas_id = IRIS_PAS_ID, + .tz_cp_config_data = &tz_cp_config_sm8250, }; diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c index 0db832b099ac..1bb34c3e6e18 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c @@ -25,6 +25,13 @@ static const struct platform_clk_data sm8550_clk_table[] = { {IRIS_HW_CLK, "vcodec0_core" }, }; +static struct tz_cp_config tz_cp_config_sm8550 = { + .cp_start = 0, + .cp_size = 0x25800000, + .cp_nonpixel_start = 0x01000000, + .cp_nonpixel_size = 0x24800000, +}; + struct iris_platform_data sm8550_data = { .get_instance = iris_hfi_gen2_get_instance, .icc_tbl = sm8550_icc_table, @@ -38,4 +45,7 @@ struct iris_platform_data sm8550_data = { .clk_tbl = sm8550_clk_table, .clk_tbl_size = ARRAY_SIZE(sm8550_clk_table), .dma_mask = GENMASK(31, 29) - 1, + .fwname = "qcom/vpu/vpu30_p4.mbn", + .pas_id = IRIS_PAS_ID, + .tz_cp_config_data = &tz_cp_config_sm8550, }; From patchwork Tue Aug 27 10:05:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 824191 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 718F219EED6; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; cv=none; b=mUaJWX6ix4lOmAfS2CB9tNCugkXtZsmH1CzYh2APvlfi4w8Y2/0jJb+EhqRSQhqP6zTXaNreMvfIOms6ZKiUAtLCFri6p9c5OyW5W0nzq72UatfDwyrEZ2no8thdCASTxRxJwTMz/gMV11RNNr03W0x5yb9AxnYjFPOdRlW7KF0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; c=relaxed/simple; bh=dGedGxPef2BEMiILioXsvSiip310l/fl+IFf11PTqsU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=AyVIuKTGx4uSQadEepiGp4WX0FEcZ/0+KxOOver3SrsSnb56VyLefpctUGcDtj43G0q7yQkzKfXw8NV9SpXYkEx+GCbcjxU3DlSwP/w15GjsrKUsc9TLYYzJxwwKEy5J44LFqmm4+BejHKDvZCrk0I67U/4rwHYaGemibqX7KZ4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=AAyCGBAn; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="AAyCGBAn" Received: by smtp.kernel.org (Postfix) with ESMTPS id CEFD1C4AF13; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753194; bh=dGedGxPef2BEMiILioXsvSiip310l/fl+IFf11PTqsU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=AAyCGBAnutdFxfChLx94lPz03v8RlzGl8iJmItSaOF+y3i54ipQLdjf61xnS9OXEK /CDAwEohp/d1s7plucbfibo7HnN8XT6D6lVpAFYysS7SK2oATn0t6f6UDzf0jZzV6z vE8Cw8QKd8kKbq6Z+dGf9qVsYkheEvnF00ZfukmEriyAAXh8G8U8FTlO4r+k02n/+l d2urto0j/eQAApr8HaHuj5rkAu+dfou0tadv/yvZ4bYqrqjL5AO1LyWWPi8YYAtPYc vdZTu7msoNsBNurrYN0FmlwJAOZrquNOUAAuP9VbwONqV7UN8vDIf/8mxaah9hgQen bxdGomjmnu2tA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id C1D2EC52D6F; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:33 +0530 Subject: [PATCH v3 08/29] media: iris: implement boot sequence of the firmware Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-8-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=6375; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=vprjPkfA56WiJKSKCvvkhoItc/1c1Wiqg2FkMZE2yYw=; b=uVGCxI/YEeoYsgKcYehxlphcA2joA23fV2m4CrMOgBwmxgSANx12Jaz/12FmdfI3Yv8AMwgrL 9YqihN/UNR9B3eNWYrfUkrnytLvoRMVpBIWGFcT3LZT1OYo7ZDcKBqV X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal Set memory region to firmware and implement boot sequence. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/Makefile | 1 + drivers/media/platform/qcom/iris/iris_core.c | 7 ++ .../platform/qcom/iris/iris_platform_common.h | 1 + .../platform/qcom/iris/iris_platform_sm8550.c | 3 + drivers/media/platform/qcom/iris/iris_vpu_common.c | 87 ++++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_vpu_common.h | 13 ++++ 6 files changed, 112 insertions(+) diff --git a/drivers/media/platform/qcom/iris/Makefile b/drivers/media/platform/qcom/iris/Makefile index ddd4c994a0b9..95f4e92fe085 100644 --- a/drivers/media/platform/qcom/iris/Makefile +++ b/drivers/media/platform/qcom/iris/Makefile @@ -8,5 +8,6 @@ iris-objs += iris_core.o \ iris_probe.o \ iris_resources.o \ iris_vidc.o \ + iris_vpu_common.o \ obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/iris/iris_core.c b/drivers/media/platform/qcom/iris/iris_core.c index 8c7d53c57086..5ad66ac113ae 100644 --- a/drivers/media/platform/qcom/iris/iris_core.c +++ b/drivers/media/platform/qcom/iris/iris_core.c @@ -6,6 +6,7 @@ #include "iris_core.h" #include "iris_firmware.h" #include "iris_state.h" +#include "iris_vpu_common.h" void iris_core_deinit(struct iris_core *core) { @@ -39,10 +40,16 @@ int iris_core_init(struct iris_core *core) if (ret) goto error_queue_deinit; + ret = iris_vpu_boot_firmware(core); + if (ret) + goto error_unload_fw; + mutex_unlock(&core->lock); return 0; +error_unload_fw: + iris_fw_unload(core); error_queue_deinit: iris_hfi_queues_deinit(core); error: diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h index 9c919367f9d7..47fdebd8135c 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_common.h +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h @@ -45,6 +45,7 @@ struct iris_platform_data { const char *fwname; u32 pas_id; struct tz_cp_config *tz_cp_config_data; + u32 core_arch; }; #endif diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c index 1bb34c3e6e18..a559e095fefc 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c @@ -8,6 +8,8 @@ #include "iris_platform_common.h" #include "iris_resources.h" +#define VIDEO_ARCH_LX 1 + static const struct icc_info sm8550_icc_table[] = { { "cpu-cfg", 1000, 1000 }, { "video-mem", 1000, 15000000 }, @@ -48,4 +50,5 @@ struct iris_platform_data sm8550_data = { .fwname = "qcom/vpu/vpu30_p4.mbn", .pas_id = IRIS_PAS_ID, .tz_cp_config_data = &tz_cp_config_sm8550, + .core_arch = VIDEO_ARCH_LX, }; diff --git a/drivers/media/platform/qcom/iris/iris_vpu_common.c b/drivers/media/platform/qcom/iris/iris_vpu_common.c new file mode 100644 index 000000000000..df87b1b719a9 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_vpu_common.c @@ -0,0 +1,87 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include + +#include "iris_core.h" +#include "iris_vpu_common.h" + +#define CPU_BASE_OFFS 0x000A0000 + +#define CPU_CS_BASE_OFFS (CPU_BASE_OFFS) + +#define CTRL_INIT (CPU_CS_BASE_OFFS + 0x48) +#define CTRL_STATUS (CPU_CS_BASE_OFFS + 0x4C) + +#define CTRL_ERROR_STATUS__M 0xfe + +#define QTBL_INFO (CPU_CS_BASE_OFFS + 0x50) +#define QTBL_ADDR (CPU_CS_BASE_OFFS + 0x54) +#define CPU_CS_SCIACMDARG3 (CPU_CS_BASE_OFFS + 0x58) +#define SFR_ADDR (CPU_CS_BASE_OFFS + 0x5C) +#define UC_REGION_ADDR (CPU_CS_BASE_OFFS + 0x64) +#define UC_REGION_SIZE (CPU_CS_BASE_OFFS + 0x68) + +#define CPU_CS_H2XSOFTINTEN (CPU_CS_BASE_OFFS + 0x148) +#define CPU_CS_X2RPMH (CPU_CS_BASE_OFFS + 0x168) + +static void iris_vpu_setup_ucregion_memory_map(struct iris_core *core) +{ + u32 queue_size, value; + + /* Iris hardware requires 4K queue alignment */ + queue_size = ALIGN(sizeof(struct iris_hfi_queue_table_header) + + (IFACEQ_QUEUE_SIZE * IFACEQ_NUMQ), SZ_4K); + + value = (u32)core->iface_q_table_daddr; + writel(value, core->reg_base + UC_REGION_ADDR); + + /* Iris hardware requires 1M queue alignment */ + value = ALIGN(SFR_SIZE + queue_size, SZ_1M); + writel(value, core->reg_base + UC_REGION_SIZE); + + value = (u32)core->iface_q_table_daddr; + writel(value, core->reg_base + QTBL_ADDR); + + writel(0x01, core->reg_base + QTBL_INFO); + + if (core->sfr_daddr) { + value = (u32)core->sfr_daddr + core->iris_platform_data->core_arch; + writel(value, core->reg_base + SFR_ADDR); + } +} + +int iris_vpu_boot_firmware(struct iris_core *core) +{ + u32 ctrl_init = 0, ctrl_status = 0, count = 0, max_tries = 1000; + + iris_vpu_setup_ucregion_memory_map(core); + + ctrl_init = BIT(0); + + writel(ctrl_init, core->reg_base + CTRL_INIT); + writel(0x1, core->reg_base + CPU_CS_SCIACMDARG3); + + while (!ctrl_status && count < max_tries) { + ctrl_status = readl(core->reg_base + CTRL_STATUS); + if ((ctrl_status & CTRL_ERROR_STATUS__M) == 0x4) { + dev_err(core->dev, "invalid setting for uc_region\n"); + break; + } + + usleep_range(50, 100); + count++; + } + + if (count >= max_tries) { + dev_err(core->dev, "error booting up iris firmware\n"); + return -ETIME; + } + + writel(0x1, core->reg_base + CPU_CS_H2XSOFTINTEN); + writel(0x0, core->reg_base + CPU_CS_X2RPMH); + + return 0; +} diff --git a/drivers/media/platform/qcom/iris/iris_vpu_common.h b/drivers/media/platform/qcom/iris/iris_vpu_common.h new file mode 100644 index 000000000000..d9b8df6e3f80 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_vpu_common.h @@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_VPU_COMMON_H_ +#define _IRIS_VPU_COMMON_H_ + +struct iris_core; + +int iris_vpu_boot_firmware(struct iris_core *core); + +#endif From patchwork Tue Aug 27 10:05:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 824192 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 22FF119E7E7; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; cv=none; b=pBtHiFQM3GV68j9GTPjd/hQpzgcfUbFfjjcQoWt9An/jnDwWCJVE4aMMYyK2SsK6NwcD6NoSG649W3X3TLB5noqHEmxXczIiVvafOSupNSMPCGI0267b29CmjStQJevt9yjvNHu75OxNVd8a8bOGJgOxuHf9tcStgQ2osEBnuDI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; c=relaxed/simple; bh=UVNJ71Ij9oExCG/XltWUXJz2Zs49OHsINOrY5m5kMMc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=p5sYqSSO/Bwoh5lKfQFIetj0QTebBzVXupcaI3X5U/4bd161r5iCTaakXw9+ET7AfpPJ4OE6o/4r3OQNitjJXd4TddFkJtEcJ1Re4ju6IxXKmV6JaOVVSRhHk44+9ctQQw5CAwJaa7QL1LQ44N14fISVCkrhY4pqCNjpvhQY6gc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Lu/DdtrJ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Lu/DdtrJ" Received: by smtp.kernel.org (Postfix) with ESMTPS id E846FC4DDEA; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753195; bh=UVNJ71Ij9oExCG/XltWUXJz2Zs49OHsINOrY5m5kMMc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=Lu/DdtrJGiGjk5KWBzbxXiZlgImFtkmhaAYkEJ/eksWisc9RALpnfneLstBPIc1V4 NX2ZrJP2yxBCqJf+9lk0XnzWXadgSawdhBKLIpbqwXYRw7el1XF0WqU11O3tWRKKAr IHD+wpS0qVU/uiYFjR7JhVIiuUHnXexFLDL7ob8xCEGzas3LVfhMKm5N8cHeV6Rciz fksFbC8TQpOrkSEHFZcfozPmQ2Q1QfhNZHjhS9ivMSOj/UOJXqHJcEYMqOIuIbdvv7 VkjCGdd1JqAvDXTN6AwJnXl0gkg85u7l6Yx7CwWj0cMSHo/ommt8xyAYbY7JXTP3D2 mBI1W7H474YCA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD6FCC5472E; Tue, 27 Aug 2024 10:06:34 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:34 +0530 Subject: [PATCH v3 09/29] media: iris: introduce Host firmware interface with necessary hooks Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-9-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=54416; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=4+G1OVfTsQ/pXuuP7tdqHzCEh/3AP4KeyYOvtsL+52w=; b=om+BThK/QE2oUffnPD8rreMXMyMVofwh3JVr0Wr46FyiTzNzEVhgI2FMMPBjp/ZQ7a9/04BKy vX8OaPikwmDDfh1xZe7WyeEhqNNAl0SNqCkR1MKIXGDUO/BkwdvSdAd X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal Host firmware interface (HFI) is well defined set of interfaces for communication between host driver and firmware. The command and responses are exchanged in form of packets. One or multiple packets are grouped under packet header. Each packet has packet type which describes the specific HFI and payload which holds the corresponding value for that HFI. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/Makefile | 5 + drivers/media/platform/qcom/iris/iris_core.c | 26 ++- drivers/media/platform/qcom/iris/iris_core.h | 20 ++ drivers/media/platform/qcom/iris/iris_hfi_common.c | 56 +++++ drivers/media/platform/qcom/iris/iris_hfi_common.h | 60 ++++++ drivers/media/platform/qcom/iris/iris_hfi_gen1.h | 3 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 61 ++++++ .../platform/qcom/iris/iris_hfi_gen1_defines.h | 94 +++++++++ .../platform/qcom/iris/iris_hfi_gen1_response.c | 174 ++++++++++++++++ drivers/media/platform/qcom/iris/iris_hfi_gen2.h | 4 + .../platform/qcom/iris/iris_hfi_gen2_command.c | 72 +++++++ .../platform/qcom/iris/iris_hfi_gen2_defines.h | 46 +++++ .../platform/qcom/iris/iris_hfi_gen2_packet.c | 164 +++++++++++++++ .../platform/qcom/iris/iris_hfi_gen2_packet.h | 69 +++++++ .../platform/qcom/iris/iris_hfi_gen2_response.c | 229 +++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_hfi_queue.c | 201 ++++++++++++++++++ drivers/media/platform/qcom/iris/iris_hfi_queue.h | 5 + .../platform/qcom/iris/iris_platform_common.h | 15 ++ .../platform/qcom/iris/iris_platform_sm8250.c | 3 + .../platform/qcom/iris/iris_platform_sm8550.c | 14 ++ drivers/media/platform/qcom/iris/iris_probe.c | 40 ++++ drivers/media/platform/qcom/iris/iris_state.c | 15 ++ drivers/media/platform/qcom/iris/iris_state.h | 4 + drivers/media/platform/qcom/iris/iris_vpu_common.c | 42 ++++ drivers/media/platform/qcom/iris/iris_vpu_common.h | 3 + 25 files changed, 1424 insertions(+), 1 deletion(-) diff --git a/drivers/media/platform/qcom/iris/Makefile b/drivers/media/platform/qcom/iris/Makefile index 95f4e92fe085..d1f0b933df3d 100644 --- a/drivers/media/platform/qcom/iris/Makefile +++ b/drivers/media/platform/qcom/iris/Makefile @@ -1,12 +1,17 @@ iris-objs += iris_core.o \ iris_firmware.o \ + iris_hfi_common.o \ iris_hfi_gen1_command.o \ + iris_hfi_gen1_response.o \ iris_hfi_gen2_command.o \ + iris_hfi_gen2_packet.o \ + iris_hfi_gen2_response.o \ iris_hfi_queue.o \ iris_platform_sm8250.o \ iris_platform_sm8550.o \ iris_probe.o \ iris_resources.o \ + iris_state.o \ iris_vidc.o \ iris_vpu_common.o \ diff --git a/drivers/media/platform/qcom/iris/iris_core.c b/drivers/media/platform/qcom/iris/iris_core.c index 5ad66ac113ae..92458d7f1e36 100644 --- a/drivers/media/platform/qcom/iris/iris_core.c +++ b/drivers/media/platform/qcom/iris/iris_core.c @@ -17,6 +17,26 @@ void iris_core_deinit(struct iris_core *core) mutex_unlock(&core->lock); } +static int iris_wait_for_system_response(struct iris_core *core) +{ + u32 hw_response_timeout_val; + int ret; + + if (core->state == IRIS_CORE_ERROR) + return -EIO; + + hw_response_timeout_val = core->iris_platform_data->hw_response_timeout; + + ret = wait_for_completion_timeout(&core->core_init_done, + msecs_to_jiffies(hw_response_timeout_val)); + if (!ret) { + iris_change_core_state(core, IRIS_CORE_ERROR); + return -ETIMEDOUT; + } + + return 0; +} + int iris_core_init(struct iris_core *core) { int ret; @@ -44,9 +64,13 @@ int iris_core_init(struct iris_core *core) if (ret) goto error_unload_fw; + ret = iris_hfi_core_init(core); + if (ret) + goto error_unload_fw; + mutex_unlock(&core->lock); - return 0; + return iris_wait_for_system_response(core); error_unload_fw: iris_fw_unload(core); diff --git a/drivers/media/platform/qcom/iris/iris_core.h b/drivers/media/platform/qcom/iris/iris_core.h index 13c5932f9110..409f9822807d 100644 --- a/drivers/media/platform/qcom/iris/iris_core.h +++ b/drivers/media/platform/qcom/iris/iris_core.h @@ -9,11 +9,15 @@ #include #include +#include "iris_hfi_common.h" #include "iris_hfi_queue.h" #include "iris_platform_common.h" #include "iris_resources.h" #include "iris_state.h" +#define IRIS_FW_VERSION_LENGTH 128 +#define IFACEQ_CORE_PKT_SIZE (1024 * 4) + /** * struct iris_core - holds core parameters valid for all instances * @@ -40,6 +44,14 @@ * @message_queue: shared interface queue to receive responses from firmware * @debug_queue: shared interface queue to receive debug info from firmware * @lock: a lock for this strucure + * @response_packet: a pointer to response packet from fw to driver + * @header_id: id of packet header + * @packet_id: id of packet + * @hfi_ops: iris hfi command ops + * @hfi_response_ops: iris hfi response ops + * @core_init_done: structure of signal completion for system response + * @intr_status: interrupt status + * @sys_error_handler: a delayed work for handling system fatal error */ struct iris_core { @@ -66,6 +78,14 @@ struct iris_core { struct iris_iface_q_info message_queue; struct iris_iface_q_info debug_queue; struct mutex lock; /* lock for core related operations */ + u8 *response_packet; + u32 header_id; + u32 packet_id; + const struct iris_hfi_command_ops *hfi_ops; + const struct iris_hfi_response_ops *hfi_response_ops; + struct completion core_init_done; + u32 intr_status; + struct delayed_work sys_error_handler; }; int iris_core_init(struct iris_core *core); diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.c b/drivers/media/platform/qcom/iris/iris_hfi_common.c new file mode 100644 index 000000000000..a5a28029d8d1 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_hfi_common.c @@ -0,0 +1,56 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_core.h" +#include "iris_hfi_common.h" +#include "iris_vpu_common.h" + +int iris_hfi_core_init(struct iris_core *core) +{ + const struct iris_hfi_command_ops *hfi_ops = core->hfi_ops; + int ret; + + ret = hfi_ops->sys_init(core); + if (ret) + return ret; + + ret = hfi_ops->sys_image_version(core); + if (ret) + return ret; + + return hfi_ops->sys_interframe_powercollapse(core); +} + +irqreturn_t iris_hfi_isr(int irq, void *data) +{ + disable_irq_nosync(irq); + + return IRQ_WAKE_THREAD; +} + +irqreturn_t iris_hfi_isr_handler(int irq, void *data) +{ + struct iris_core *core = data; + + if (!core) + return IRQ_NONE; + + mutex_lock(&core->lock); + if (core->state != IRIS_CORE_INIT) { + mutex_unlock(&core->lock); + goto exit; + } + + iris_vpu_clear_interrupt(core); + mutex_unlock(&core->lock); + + core->hfi_response_ops->hfi_response_handler(core); + +exit: + if (!iris_vpu_watchdog(core, core->intr_status)) + enable_irq(irq); + + return IRQ_HANDLED; +} diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.h b/drivers/media/platform/qcom/iris/iris_hfi_common.h new file mode 100644 index 000000000000..c3d5b899cf60 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_hfi_common.h @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_HFI_COMMON_H_ +#define _IRIS_HFI_COMMON_H_ + +#include +#include + +struct iris_core; + +enum hfi_packet_port_type { + HFI_PORT_NONE = 0x00000000, + HFI_PORT_BITSTREAM = 0x00000001, + HFI_PORT_RAW = 0x00000002, +}; + +enum hfi_packet_payload_info { + HFI_PAYLOAD_NONE = 0x00000000, + HFI_PAYLOAD_U32 = 0x00000001, + HFI_PAYLOAD_S32 = 0x00000002, + HFI_PAYLOAD_U64 = 0x00000003, + HFI_PAYLOAD_S64 = 0x00000004, + HFI_PAYLOAD_STRUCTURE = 0x00000005, + HFI_PAYLOAD_BLOB = 0x00000006, + HFI_PAYLOAD_STRING = 0x00000007, + HFI_PAYLOAD_Q16 = 0x00000008, + HFI_PAYLOAD_U32_ENUM = 0x00000009, + HFI_PAYLOAD_32_PACKED = 0x0000000a, + HFI_PAYLOAD_U32_ARRAY = 0x0000000b, + HFI_PAYLOAD_S32_ARRAY = 0x0000000c, + HFI_PAYLOAD_64_PACKED = 0x0000000d, +}; + +enum hfi_packet_host_flags { + HFI_HOST_FLAGS_NONE = 0x00000000, + HFI_HOST_FLAGS_INTR_REQUIRED = 0x00000001, + HFI_HOST_FLAGS_RESPONSE_REQUIRED = 0x00000002, + HFI_HOST_FLAGS_NON_DISCARDABLE = 0x00000004, + HFI_HOST_FLAGS_GET_PROPERTY = 0x00000008, +}; + +struct iris_hfi_command_ops { + int (*sys_init)(struct iris_core *core); + int (*sys_image_version)(struct iris_core *core); + int (*sys_interframe_powercollapse)(struct iris_core *core); +}; + +struct iris_hfi_response_ops { + void (*hfi_response_handler)(struct iris_core *core); +}; + +int iris_hfi_core_init(struct iris_core *core); + +irqreturn_t iris_hfi_isr(int irq, void *data); +irqreturn_t iris_hfi_isr_handler(int irq, void *data); + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1.h index b02f629a9cdc..15edbb359c71 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1.h @@ -6,8 +6,11 @@ #ifndef _IRIS_HFI_GEN1_H_ #define _IRIS_HFI_GEN1_H_ +struct iris_core; struct iris_inst; +void iris_hfi_gen1_command_ops_init(struct iris_core *core); +void iris_hfi_gen1_response_ops_init(struct iris_core *core); struct iris_inst *iris_hfi_gen1_get_instance(void); #endif diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c index 20c68f4ffb72..8f045ef56163 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c @@ -4,8 +4,69 @@ */ #include "iris_hfi_gen1.h" +#include "iris_hfi_gen1_defines.h" #include "iris_instance.h" +static int iris_hfi_gen1_sys_init(struct iris_core *core) +{ + struct hfi_sys_init_pkt sys_init_pkt; + + sys_init_pkt.hdr.size = sizeof(sys_init_pkt); + sys_init_pkt.hdr.pkt_type = HFI_CMD_SYS_INIT; + sys_init_pkt.arch_type = HFI_VIDEO_ARCH_OX; + + return iris_hfi_queue_cmd_write_locked(core, &sys_init_pkt, sys_init_pkt.hdr.size); +} + +static int iris_hfi_gen1_sys_image_version(struct iris_core *core) +{ + struct hfi_sys_get_property_pkt packet; + + packet.hdr.size = sizeof(packet); + packet.hdr.pkt_type = HFI_CMD_SYS_GET_PROPERTY; + packet.num_properties = 1; + packet.data = HFI_PROPERTY_SYS_IMAGE_VERSION; + + return iris_hfi_queue_cmd_write_locked(core, &packet, packet.hdr.size); +} + +static int iris_hfi_gen1_sys_interframe_powercollapse(struct iris_core *core) +{ + struct hfi_sys_set_property_pkt *pkt; + struct hfi_enable *hfi; + u32 packet_size; + u32 ret; + + packet_size = struct_size(pkt, data, 1) + sizeof(*hfi); + pkt = kzalloc(packet_size, GFP_KERNEL); + if (!pkt) + return -ENOMEM; + + hfi = (struct hfi_enable *)&pkt->data[1]; + + pkt->hdr.size = packet_size; + pkt->hdr.pkt_type = HFI_CMD_SYS_SET_PROPERTY; + pkt->num_properties = 1; + pkt->data[0] = HFI_PROPERTY_SYS_CODEC_POWER_PLANE_CTRL; + hfi->enable = true; + + ret = iris_hfi_queue_cmd_write_locked(core, pkt, pkt->hdr.size); + kfree(pkt); + + return ret; +} + +static const struct iris_hfi_command_ops iris_hfi_gen1_command_ops = { + .sys_init = iris_hfi_gen1_sys_init, + .sys_image_version = iris_hfi_gen1_sys_image_version, + .sys_interframe_powercollapse = iris_hfi_gen1_sys_interframe_powercollapse, +}; + +void iris_hfi_gen1_command_ops_init(struct iris_core *core) +{ + core->hfi_ops = &iris_hfi_gen1_command_ops; +} + struct iris_inst *iris_hfi_gen1_get_instance(void) { return kzalloc(sizeof(struct iris_inst), GFP_KERNEL); diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h new file mode 100644 index 000000000000..5c07d6a29863 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h @@ -0,0 +1,94 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_HFI_GEN1_DEFINES_H_ +#define _IRIS_HFI_GEN1_DEFINES_H_ + +#include + +#define HFI_VIDEO_ARCH_OX 0x1 +#define HFI_ERR_NONE 0x0 + +#define HFI_CMD_SYS_INIT 0x10001 +#define HFI_CMD_SYS_SET_PROPERTY 0x10005 +#define HFI_CMD_SYS_GET_PROPERTY 0x10006 + +#define HFI_PROPERTY_SYS_CODEC_POWER_PLANE_CTRL 0x5 +#define HFI_PROPERTY_SYS_IMAGE_VERSION 0x6 + +#define HFI_EVENT_SYS_ERROR 0x1 + +#define HFI_MSG_SYS_INIT 0x20001 +#define HFI_MSG_SYS_COV 0x20009 +#define HFI_MSG_SYS_PROPERTY_INFO 0x2000a + +#define HFI_MSG_EVENT_NOTIFY 0x21001 + +struct hfi_pkt_hdr { + u32 size; + u32 pkt_type; +}; + +struct hfi_sys_init_pkt { + struct hfi_pkt_hdr hdr; + u32 arch_type; +}; + +struct hfi_sys_set_property_pkt { + struct hfi_pkt_hdr hdr; + u32 num_properties; + u32 data[]; +}; + +struct hfi_sys_get_property_pkt { + struct hfi_pkt_hdr hdr; + u32 num_properties; + u32 data; +}; + +struct hfi_msg_event_notify_pkt { + struct hfi_pkt_hdr hdr; + u32 event_id; + u32 event_data1; + u32 event_data2; + u32 ext_event_data[]; +}; + +struct hfi_msg_sys_init_done_pkt { + struct hfi_pkt_hdr hdr; + u32 error_type; + u32 num_properties; + u32 data[]; +}; + +struct hfi_msg_sys_property_info_pkt { + struct hfi_pkt_hdr hdr; + u32 num_properties; + u32 property; + u8 data[]; +}; + +struct hfi_enable { + u32 enable; +}; + +struct hfi_msg_sys_debug_pkt { + struct hfi_pkt_hdr hdr; + u32 msg_type; + u32 msg_size; + u32 time_stamp_hi; + u32 time_stamp_lo; + u8 msg_data[]; +}; + +struct hfi_msg_sys_coverage_pkt { + struct hfi_pkt_hdr hdr; + u32 msg_size; + u32 time_stamp_hi; + u32 time_stamp_lo; + u8 msg_data[]; +}; + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c new file mode 100644 index 000000000000..3eb2ce99c614 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c @@ -0,0 +1,174 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_hfi_gen1.h" +#include "iris_hfi_gen1_defines.h" +#include "iris_instance.h" + +static void +iris_hfi_gen1_sys_event_notify(struct iris_core *core, void *packet) +{ + struct hfi_msg_event_notify_pkt *pkt = packet; + + if (pkt->event_id == HFI_EVENT_SYS_ERROR) + dev_err(core->dev, "sys error (type: %x, data1:%x, data2:%x)\n", + pkt->event_id, pkt->event_data1, pkt->event_data2); + + iris_change_core_state(core, IRIS_CORE_ERROR); + schedule_delayed_work(&core->sys_error_handler, msecs_to_jiffies(10)); +} + +static void iris_hfi_gen1_sys_init_done(struct iris_core *core, void *packet) +{ + struct hfi_msg_sys_init_done_pkt *pkt = packet; + + if (pkt->error_type != HFI_ERR_NONE) { + iris_change_core_state(core, IRIS_CORE_ERROR); + return; + } + + complete(&core->core_init_done); +} + +static void +iris_hfi_gen1_sys_get_prop_image_version(struct iris_core *core, + struct hfi_msg_sys_property_info_pkt *pkt) +{ + char fw_version[IRIS_FW_VERSION_LENGTH]; + u8 *str_image_version; + int req_bytes; + u32 i; + + req_bytes = pkt->hdr.size - sizeof(*pkt); + + if (req_bytes < IRIS_FW_VERSION_LENGTH - 1 || !pkt->data[0] || pkt->num_properties > 1) + /* bad packet */ + return; + + str_image_version = pkt->data; + if (!str_image_version) + return; + + for (i = 0; i < IRIS_FW_VERSION_LENGTH - 1; i++) { + if (str_image_version[i] != '\0') + fw_version[i] = str_image_version[i]; + else + fw_version[i] = ' '; + } + fw_version[i] = '\0'; + + dev_dbg(core->dev, "firmware version: %s\n", fw_version); +} + +static void iris_hfi_gen1_sys_property_info(struct iris_core *core, void *packet) +{ + struct hfi_msg_sys_property_info_pkt *pkt = packet; + + if (!pkt->num_properties) { + dev_dbg(core->dev, "no properties\n"); + return; + } + + switch (pkt->property) { + case HFI_PROPERTY_SYS_IMAGE_VERSION: + iris_hfi_gen1_sys_get_prop_image_version(core, pkt); + break; + default: + dev_dbg(core->dev, "unknown property data\n"); + break; + } +} + +struct iris_hfi_gen1_response_pkt_info { + u32 pkt; + u32 pkt_sz; +}; + +static const struct iris_hfi_gen1_response_pkt_info pkt_infos[] = { + { + .pkt = HFI_MSG_EVENT_NOTIFY, + .pkt_sz = sizeof(struct hfi_msg_event_notify_pkt), + }, + { + .pkt = HFI_MSG_SYS_INIT, + .pkt_sz = sizeof(struct hfi_msg_sys_init_done_pkt), + }, + { + .pkt = HFI_MSG_SYS_PROPERTY_INFO, + .pkt_sz = sizeof(struct hfi_msg_sys_property_info_pkt), + }, +}; + +static void iris_hfi_gen1_handle_response(struct iris_core *core, void *response) +{ + const struct iris_hfi_gen1_response_pkt_info *pkt_info; + struct device *dev = core->dev; + struct hfi_pkt_hdr *hdr; + bool found = false; + unsigned int i; + + hdr = (struct hfi_pkt_hdr *)response; + + for (i = 0; i < ARRAY_SIZE(pkt_infos); i++) { + pkt_info = &pkt_infos[i]; + if (pkt_info->pkt != hdr->pkt_type) + continue; + found = true; + break; + } + + if (!found || hdr->size < pkt_info->pkt_sz) { + dev_err(dev, "bad packet size (%d should be %d, pkt type:%x, found %d)\n", + hdr->size, pkt_info->pkt_sz, hdr->pkt_type, found); + + return; + } + + if (hdr->pkt_type == HFI_MSG_SYS_INIT) + iris_hfi_gen1_sys_init_done(core, hdr); + else if (hdr->pkt_type == HFI_MSG_SYS_PROPERTY_INFO) + iris_hfi_gen1_sys_property_info(core, hdr); + else if (hdr->pkt_type == HFI_MSG_EVENT_NOTIFY) + iris_hfi_gen1_sys_event_notify(core, hdr); +} + +static void iris_hfi_gen1_flush_debug_queue(struct iris_core *core, u8 *packet) +{ + struct hfi_msg_sys_coverage_pkt *pkt; + + while (!iris_hfi_queue_dbg_read(core, packet)) { + pkt = (struct hfi_msg_sys_coverage_pkt *)packet; + + if (pkt->hdr.pkt_type != HFI_MSG_SYS_COV) { + struct hfi_msg_sys_debug_pkt *pkt = + (struct hfi_msg_sys_debug_pkt *)packet; + + dev_dbg(core->dev, "%s", pkt->msg_data); + } + } +} + +static void iris_hfi_gen1_response_handler(struct iris_core *core) +{ + memset(core->response_packet, 0, sizeof(struct hfi_pkt_hdr)); + while (!iris_hfi_queue_msg_read(core, core->response_packet)) { + iris_hfi_gen1_handle_response(core, core->response_packet); + if (core->state != IRIS_CORE_INIT) + break; + + memset(core->response_packet, 0, sizeof(struct hfi_pkt_hdr)); + } + + iris_hfi_gen1_flush_debug_queue(core, core->response_packet); +} + +static const struct iris_hfi_response_ops iris_hfi_gen1_response_ops = { + .hfi_response_handler = iris_hfi_gen1_response_handler, +}; + +void iris_hfi_gen1_response_ops_init(struct iris_core *core) +{ + core->hfi_response_ops = &iris_hfi_gen1_response_ops; +} diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2.h index 4f9748cbe0e3..6ec83984fda9 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2.h @@ -8,6 +8,8 @@ #include "iris_instance.h" +struct iris_core; + /** * struct iris_inst_hfi_gen2 - holds per video instance parameters for hfi_gen2 * @@ -17,6 +19,8 @@ struct iris_inst_hfi_gen2 { struct iris_inst inst; }; +void iris_hfi_gen2_command_ops_init(struct iris_core *core); +void iris_hfi_gen2_response_ops_init(struct iris_core *core); struct iris_inst *iris_hfi_gen2_get_instance(void); #endif diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c index 3ee33c8befae..807266858d93 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c @@ -4,6 +4,78 @@ */ #include "iris_hfi_gen2.h" +#include "iris_hfi_gen2_packet.h" + +#define NUM_SYS_INIT_PACKETS 8 + +static int iris_hfi_gen2_sys_init(struct iris_core *core) +{ + struct iris_hfi_header *hdr; + u32 packet_size; + int ret; + + packet_size = sizeof(*hdr) + + NUM_SYS_INIT_PACKETS * (sizeof(struct iris_hfi_packet) + sizeof(u32)); + hdr = kzalloc(packet_size, GFP_KERNEL); + if (!hdr) + return -ENOMEM; + + iris_hfi_gen2_packet_sys_init(core, hdr); + ret = iris_hfi_queue_cmd_write_locked(core, hdr, hdr->size); + + kfree(hdr); + + return ret; +} + +static int iris_hfi_gen2_sys_image_version(struct iris_core *core) +{ + struct iris_hfi_header *hdr; + u32 packet_size; + int ret; + + packet_size = sizeof(*hdr) + sizeof(struct iris_hfi_packet); + hdr = kzalloc(packet_size, GFP_KERNEL); + if (!hdr) + return -ENOMEM; + + iris_hfi_gen2_packet_image_version(core, hdr); + ret = iris_hfi_queue_cmd_write_locked(core, hdr, hdr->size); + + kfree(hdr); + + return ret; +} + +static int iris_hfi_gen2_sys_interframe_powercollapse(struct iris_core *core) +{ + struct iris_hfi_header *hdr; + u32 packet_size; + int ret; + + packet_size = sizeof(*hdr) + sizeof(struct iris_hfi_packet) + sizeof(u32); + hdr = kzalloc(packet_size, GFP_KERNEL); + if (!hdr) + return -ENOMEM; + + iris_hfi_gen2_packet_sys_interframe_powercollapse(core, hdr); + ret = iris_hfi_queue_cmd_write_locked(core, hdr, hdr->size); + + kfree(hdr); + + return ret; +} + +static const struct iris_hfi_command_ops iris_hfi_gen2_command_ops = { + .sys_init = iris_hfi_gen2_sys_init, + .sys_image_version = iris_hfi_gen2_sys_image_version, + .sys_interframe_powercollapse = iris_hfi_gen2_sys_interframe_powercollapse, +}; + +void iris_hfi_gen2_command_ops_init(struct iris_core *core) +{ + core->hfi_ops = &iris_hfi_gen2_command_ops; +} struct iris_inst *iris_hfi_gen2_get_instance(void) { diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h new file mode 100644 index 000000000000..3e3e4ddfe21f --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h @@ -0,0 +1,46 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_HFI_GEN2_DEFINES_H_ +#define _IRIS_HFI_GEN2_DEFINES_H_ + +#include + +#define HFI_VIDEO_ARCH_LX 0x1 + +#define HFI_CMD_BEGIN 0x01000000 +#define HFI_CMD_INIT 0x01000001 +#define HFI_CMD_END 0x01FFFFFF + +#define HFI_PROP_BEGIN 0x03000000 +#define HFI_PROP_IMAGE_VERSION 0x03000001 +#define HFI_PROP_INTRA_FRAME_POWER_COLLAPSE 0x03000002 +#define HFI_PROP_UBWC_MAX_CHANNELS 0x03000003 +#define HFI_PROP_UBWC_MAL_LENGTH 0x03000004 +#define HFI_PROP_UBWC_HBB 0x03000005 +#define HFI_PROP_UBWC_BANK_SWZL_LEVEL1 0x03000006 +#define HFI_PROP_UBWC_BANK_SWZL_LEVEL2 0x03000007 +#define HFI_PROP_UBWC_BANK_SWZL_LEVEL3 0x03000008 +#define HFI_PROP_UBWC_BANK_SPREADING 0x03000009 +#define HFI_PROP_END 0x03FFFFFF + +#define HFI_SYSTEM_ERROR_BEGIN 0x05000000 +#define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001 +#define HFI_SYSTEM_ERROR_END 0x05FFFFFF + +enum hfi_packet_firmware_flags { + HFI_FW_FLAGS_SUCCESS = 0x00000001, + HFI_FW_FLAGS_INFORMATION = 0x00000002, + HFI_FW_FLAGS_SESSION_ERROR = 0x00000004, + HFI_FW_FLAGS_SYSTEM_ERROR = 0x00000008, +}; + +struct hfi_debug_header { + u32 size; + u32 debug_level; + u32 reserved[2]; +}; + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c new file mode 100644 index 000000000000..8266eae5ff94 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c @@ -0,0 +1,164 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_hfi_common.h" +#include "iris_hfi_gen2.h" +#include "iris_hfi_gen2_packet.h" + +static void iris_hfi_gen2_create_header(struct iris_hfi_header *hdr, + u32 session_id, u32 header_id) +{ + memset(hdr, 0, sizeof(*hdr)); + + hdr->size = sizeof(*hdr); + hdr->session_id = session_id; + hdr->header_id = header_id; + hdr->num_packets = 0; +} + +static void iris_hfi_gen2_create_packet(struct iris_hfi_header *hdr, u32 pkt_type, + u32 pkt_flags, u32 payload_type, u32 port, + u32 packet_id, void *payload, u32 payload_size) +{ + struct iris_hfi_packet *pkt; + u32 pkt_size; + + pkt = (struct iris_hfi_packet *)((u8 *)hdr + hdr->size); + pkt_size = sizeof(*pkt) + payload_size; + + memset(pkt, 0, pkt_size); + pkt->size = pkt_size; + pkt->type = pkt_type; + pkt->flags = pkt_flags; + pkt->payload_info = payload_type; + pkt->port = port; + pkt->packet_id = packet_id; + if (payload_size) + memcpy(&pkt->payload[0], payload, payload_size); + + hdr->num_packets++; + hdr->size += pkt->size; +} + +void iris_hfi_gen2_packet_sys_init(struct iris_core *core, struct iris_hfi_header *hdr) +{ + u32 payload = 0; + + iris_hfi_gen2_create_header(hdr, 0, core->header_id++); + + payload = HFI_VIDEO_ARCH_LX; + iris_hfi_gen2_create_packet(hdr, + HFI_CMD_INIT, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED | + HFI_HOST_FLAGS_NON_DISCARDABLE), + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); + + payload = core->iris_platform_data->ubwc_config->max_channels; + iris_hfi_gen2_create_packet(hdr, + HFI_PROP_UBWC_MAX_CHANNELS, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); + + payload = core->iris_platform_data->ubwc_config->mal_length; + iris_hfi_gen2_create_packet(hdr, + HFI_PROP_UBWC_MAL_LENGTH, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); + + payload = core->iris_platform_data->ubwc_config->highest_bank_bit; + iris_hfi_gen2_create_packet(hdr, + HFI_PROP_UBWC_HBB, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); + + payload = core->iris_platform_data->ubwc_config->bank_swzl_level; + iris_hfi_gen2_create_packet(hdr, + HFI_PROP_UBWC_BANK_SWZL_LEVEL1, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); + + payload = core->iris_platform_data->ubwc_config->bank_swz2_level; + iris_hfi_gen2_create_packet(hdr, + HFI_PROP_UBWC_BANK_SWZL_LEVEL2, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); + + payload = core->iris_platform_data->ubwc_config->bank_swz3_level; + iris_hfi_gen2_create_packet(hdr, + HFI_PROP_UBWC_BANK_SWZL_LEVEL3, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); + + payload = core->iris_platform_data->ubwc_config->bank_spreading; + iris_hfi_gen2_create_packet(hdr, + HFI_PROP_UBWC_BANK_SPREADING, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); +} + +void iris_hfi_gen2_packet_image_version(struct iris_core *core, struct iris_hfi_header *hdr) +{ + iris_hfi_gen2_create_header(hdr, 0, core->header_id++); + + iris_hfi_gen2_create_packet(hdr, + HFI_PROP_IMAGE_VERSION, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED | + HFI_HOST_FLAGS_GET_PROPERTY), + HFI_PAYLOAD_NONE, + HFI_PORT_NONE, + core->packet_id++, + NULL, 0); +} + +void iris_hfi_gen2_packet_sys_interframe_powercollapse(struct iris_core *core, + struct iris_hfi_header *hdr) +{ + u32 payload = 1; /* HFI_TRUE */ + + iris_hfi_gen2_create_header(hdr, 0 /*session_id*/, core->header_id++); + + iris_hfi_gen2_create_packet(hdr, + HFI_PROP_INTRA_FRAME_POWER_COLLAPSE, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_U32, + HFI_PORT_NONE, + core->packet_id++, + &payload, + sizeof(u32)); +} diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h new file mode 100644 index 000000000000..eba109efeb76 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_HFI_GEN2_PACKET_H_ +#define _IRIS_HFI_GEN2_PACKET_H_ + +#include "iris_hfi_gen2_defines.h" + +struct iris_core; + +/** + * struct iris_hfi_header + * + * @size: size of the total packet in bytes including hfi_header + * @session_id: For session level hfi_header session_id is non-zero. + * For system level hfi_header session_id is zero. + * @header_id: unique header id for each hfi_header + * @reserved: reserved for future use + * @num_packets: number of hfi_packet that are included with the hfi_header + */ +struct iris_hfi_header { + u32 size; + u32 session_id; + u32 header_id; + u32 reserved[4]; + u32 num_packets; +}; + +/** + * struct iris_hfi_packet + * + * @size: size of the hfi_packet in bytes including payload + * @type: one of the below hfi_packet types: + * HFI_CMD_*, + * HFI_PROP_*, + * HFI_ERROR_*, + * HFI_INFO_*, + * HFI_SYS_ERROR_* + * @flags: hfi_packet flags. It is represented as bit masks. + * host packet flags are "enum hfi_packet_host_flags" + * firmware packet flags are "enum hfi_packet_firmware_flags" + * @payload_info: payload information indicated by "enum hfi_packet_payload_info" + * @port: hfi_packet port type indicated by "enum hfi_packet_port_type" + * This is bitmask and may be applicable to multiple ports. + * @packet_id: host hfi_packet contains unique packet id. + * firmware returns host packet id in response packet + * wherever applicable. If not applicable firmware sets it to zero. + * @reserved: reserved for future use. + * @payload: flexible array of payload having additional packet information. + */ +struct iris_hfi_packet { + u32 size; + u32 type; + u32 flags; + u32 payload_info; + u32 port; + u32 packet_id; + u32 reserved[2]; + u32 payload[]; +}; + +void iris_hfi_gen2_packet_sys_init(struct iris_core *core, struct iris_hfi_header *hdr); +void iris_hfi_gen2_packet_image_version(struct iris_core *core, struct iris_hfi_header *hdr); +void iris_hfi_gen2_packet_sys_interframe_powercollapse(struct iris_core *core, + struct iris_hfi_header *hdr); + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c new file mode 100644 index 000000000000..e208a5ae664a --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c @@ -0,0 +1,229 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_hfi_gen2.h" +#include "iris_hfi_gen2_defines.h" +#include "iris_hfi_gen2_packet.h" +#include "iris_vpu_common.h" + +struct iris_hfi_gen2_core_hfi_range { + u32 begin; + u32 end; + int (*handle)(struct iris_core *core, struct iris_hfi_packet *pkt); +}; + +static int iris_hfi_gen2_validate_packet(u8 *response_pkt, u8 *core_resp_pkt) +{ + u32 response_pkt_size = 0; + u8 *response_limit; + + response_limit = core_resp_pkt + IFACEQ_CORE_PKT_SIZE; + + response_pkt_size = *(u32 *)response_pkt; + if (!response_pkt_size) + return -EINVAL; + + if (response_pkt_size < sizeof(struct iris_hfi_packet)) + return -EINVAL; + + if (response_pkt + response_pkt_size > response_limit) + return -EINVAL; + + return 0; +} + +static int iris_hfi_gen2_validate_hdr_packet(struct iris_core *core, struct iris_hfi_header *hdr) +{ + struct iris_hfi_packet *packet; + int i, ret = 0; + u8 *pkt; + + if (hdr->size < sizeof(*hdr) + sizeof(*packet)) + return -EINVAL; + + pkt = (u8 *)((u8 *)hdr + sizeof(*hdr)); + + for (i = 0; i < hdr->num_packets; i++) { + packet = (struct iris_hfi_packet *)pkt; + ret = iris_hfi_gen2_validate_packet(pkt, core->response_packet); + if (ret) + return ret; + + pkt += packet->size; + } + + return ret; +} + +static int iris_hfi_gen2_handle_system_error(struct iris_core *core, + struct iris_hfi_packet *pkt) +{ + dev_err(core->dev, "received system error of type %#x\n", pkt->type); + + iris_change_core_state(core, IRIS_CORE_ERROR); + schedule_delayed_work(&core->sys_error_handler, msecs_to_jiffies(10)); + + return 0; +} + +static int iris_hfi_gen2_handle_system_init(struct iris_core *core, + struct iris_hfi_packet *pkt) +{ + if (!(pkt->flags & HFI_FW_FLAGS_SUCCESS)) { + iris_change_core_state(core, IRIS_CORE_ERROR); + return 0; + } + + complete(&core->core_init_done); + + return 0; +} + +static int iris_hfi_gen2_handle_image_version_property(struct iris_core *core, + struct iris_hfi_packet *pkt) +{ + char fw_version[IRIS_FW_VERSION_LENGTH]; + u8 *str_image_version; + u32 req_bytes; + u32 i = 0; + + req_bytes = pkt->size - sizeof(*pkt); + if (req_bytes < IRIS_FW_VERSION_LENGTH - 1) + return -EINVAL; + + str_image_version = (u8 *)pkt + sizeof(*pkt); + + for (i = 0; i < IRIS_FW_VERSION_LENGTH - 1; i++) { + if (str_image_version[i] != '\0') + fw_version[i] = str_image_version[i]; + else + fw_version[i] = ' '; + } + fw_version[i] = '\0'; + + dev_dbg(core->dev, "firmware version: %s\n", fw_version); + + return 0; +} + +static int iris_hfi_gen2_handle_system_property(struct iris_core *core, + struct iris_hfi_packet *pkt) +{ + int ret = 0; + + switch (pkt->type) { + case HFI_PROP_IMAGE_VERSION: + ret = iris_hfi_gen2_handle_image_version_property(core, pkt); + break; + default: + break; + } + + return ret; +} + +static int iris_hfi_gen2_handle_system_response(struct iris_core *core, + struct iris_hfi_header *hdr) +{ + struct iris_hfi_packet *packet; + u8 *pkt, *start_pkt; + int ret = 0; + int i, j; + static const struct iris_hfi_gen2_core_hfi_range range[] = { + {HFI_SYSTEM_ERROR_BEGIN, HFI_SYSTEM_ERROR_END, iris_hfi_gen2_handle_system_error }, + {HFI_PROP_BEGIN, HFI_PROP_END, iris_hfi_gen2_handle_system_property }, + {HFI_CMD_BEGIN, HFI_CMD_END, iris_hfi_gen2_handle_system_init }, + }; + + start_pkt = (u8 *)((u8 *)hdr + sizeof(*hdr)); + for (i = 0; i < ARRAY_SIZE(range); i++) { + pkt = start_pkt; + for (j = 0; j < hdr->num_packets; j++) { + packet = (struct iris_hfi_packet *)pkt; + if (packet->flags & HFI_FW_FLAGS_SYSTEM_ERROR) { + ret = iris_hfi_gen2_handle_system_error(core, packet); + return ret; + } + + if (packet->type > range[i].begin && packet->type < range[i].end) { + ret = range[i].handle(core, packet); + if (ret) + return ret; + + if (packet->type > HFI_SYSTEM_ERROR_BEGIN && + packet->type < HFI_SYSTEM_ERROR_END) + return 0; + } + pkt += packet->size; + } + } + + return ret; +} + +static int iris_hfi_gen2_handle_response(struct iris_core *core, void *response) +{ + struct iris_hfi_header *hdr; + int ret; + + hdr = (struct iris_hfi_header *)response; + ret = iris_hfi_gen2_validate_hdr_packet(core, hdr); + if (ret) + return iris_hfi_gen2_handle_system_error(core, NULL); + + return iris_hfi_gen2_handle_system_response(core, hdr); +} + +static void iris_hfi_gen2_flush_debug_queue(struct iris_core *core, u8 *packet) +{ + struct hfi_debug_header *pkt; + u8 *log; + + while (!iris_hfi_queue_dbg_read(core, packet)) { + pkt = (struct hfi_debug_header *)packet; + + if (pkt->size < sizeof(*pkt)) + continue; + + if (pkt->size >= IFACEQ_CORE_PKT_SIZE) + continue; + + packet[pkt->size] = '\0'; + log = (u8 *)packet + sizeof(*pkt) + 1; + dev_dbg(core->dev, "%s", log); + } +} + +static void iris_hfi_gen2_response_handler(struct iris_core *core) +{ + if (iris_vpu_watchdog(core, core->intr_status)) { + struct iris_hfi_packet pkt = {.type = HFI_SYS_ERROR_WD_TIMEOUT}; + + dev_err(core->dev, "cpu watchdog error received\n"); + iris_change_core_state(core, IRIS_CORE_ERROR); + iris_hfi_gen2_handle_system_error(core, &pkt); + + return; + } + + memset(core->response_packet, 0, sizeof(struct iris_hfi_header)); + while (!iris_hfi_queue_msg_read(core, core->response_packet)) { + iris_hfi_gen2_handle_response(core, core->response_packet); + if (core->state != IRIS_CORE_INIT) + break; + memset(core->response_packet, 0, sizeof(struct iris_hfi_header)); + } + + iris_hfi_gen2_flush_debug_queue(core, core->response_packet); +} + +static const struct iris_hfi_response_ops iris_hfi_gen2_response_ops = { + .hfi_response_handler = iris_hfi_gen2_response_handler, +}; + +void iris_hfi_gen2_response_ops_init(struct iris_core *core) +{ + core->hfi_response_ops = &iris_hfi_gen2_response_ops; +} diff --git a/drivers/media/platform/qcom/iris/iris_hfi_queue.c b/drivers/media/platform/qcom/iris/iris_hfi_queue.c index 11938880b8cd..b24d4640fea9 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_queue.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_queue.c @@ -5,6 +5,207 @@ #include "iris_core.h" #include "iris_hfi_queue.h" +#include "iris_vpu_common.h" + +static int iris_hfi_queue_write(struct iris_iface_q_info *qinfo, void *packet, u32 packet_size) +{ + u32 empty_space, read_idx, write_idx, new_write_idx; + struct iris_hfi_queue_header *queue; + u32 *write_ptr; + u32 residue; + + queue = qinfo->qhdr; + + read_idx = queue->read_idx * sizeof(u32); + write_idx = queue->write_idx * sizeof(u32); + + if (write_idx < read_idx) + empty_space = read_idx - write_idx; + else + empty_space = IFACEQ_QUEUE_SIZE - (write_idx - read_idx); + if (empty_space < packet_size) + return -ENOSPC; + + queue->tx_req = 0; + + new_write_idx = write_idx + packet_size; + write_ptr = (u32 *)((u8 *)qinfo->kernel_vaddr + write_idx); + + if (write_ptr < (u32 *)qinfo->kernel_vaddr || + write_ptr > (u32 *)(qinfo->kernel_vaddr + + IFACEQ_QUEUE_SIZE)) + return -EINVAL; + + if (new_write_idx < IFACEQ_QUEUE_SIZE) { + memcpy(write_ptr, packet, packet_size); + } else { + residue = new_write_idx - IFACEQ_QUEUE_SIZE; + memcpy(write_ptr, packet, (packet_size - residue)); + memcpy(qinfo->kernel_vaddr, + packet + (packet_size - residue), residue); + new_write_idx = residue; + } + + /* Make sure packet is written before updating the write index */ + mb(); + queue->write_idx = new_write_idx / sizeof(u32); + + /* Make sure write index is updated before an interrupt is raised */ + mb(); + + return 0; +} + +static int iris_hfi_queue_read(struct iris_iface_q_info *qinfo, void *packet) +{ + u32 read_idx, write_idx, new_read_idx; + struct iris_hfi_queue_header *queue; + u32 packet_size, residue; + u32 receive_request = 0; + u32 *read_ptr; + int ret = 0; + + queue = qinfo->qhdr; + + if (queue->queue_type == IFACEQ_MSGQ_ID) + receive_request = 1; + + read_idx = queue->read_idx * sizeof(u32); + write_idx = queue->write_idx * sizeof(u32); + + if (read_idx == write_idx) { + queue->rx_req = receive_request; + /* Ensure qhdr is updated in main memory */ + mb(); + return -ENODATA; + } + + read_ptr = qinfo->kernel_vaddr + read_idx; + if (read_ptr < (u32 *)qinfo->kernel_vaddr || + read_ptr > (u32 *)(qinfo->kernel_vaddr + + IFACEQ_QUEUE_SIZE - sizeof(*read_ptr))) + return -ENODATA; + + packet_size = *read_ptr; + if (!packet_size) + return -EINVAL; + + new_read_idx = read_idx + packet_size; + if (packet_size <= IFACEQ_CORE_PKT_SIZE) { + if (new_read_idx < IFACEQ_QUEUE_SIZE) { + memcpy(packet, read_ptr, packet_size); + } else { + residue = new_read_idx - IFACEQ_QUEUE_SIZE; + memcpy(packet, read_ptr, (packet_size - residue)); + memcpy((packet + (packet_size - residue)), + qinfo->kernel_vaddr, residue); + new_read_idx = residue; + } + } else { + new_read_idx = write_idx; + ret = -EBADMSG; + } + + queue->rx_req = receive_request; + + queue->read_idx = new_read_idx / sizeof(u32); + /* Ensure qhdr is updated in main memory */ + mb(); + + return ret; +} + +int iris_hfi_queue_cmd_write_locked(struct iris_core *core, void *pkt, u32 pkt_size) +{ + struct iris_iface_q_info *q_info; + + if (!mutex_is_locked(&core->lock)) + return -EINVAL; + + if (core->state != IRIS_CORE_INIT) + return -EINVAL; + + q_info = &core->command_queue; + if (!q_info || !q_info->kernel_vaddr || !pkt) { + dev_err(core->dev, "cannot write to shared command queue\n"); + return -ENODATA; + } + + if (!iris_hfi_queue_write(q_info, pkt, pkt_size)) { + iris_vpu_raise_interrupt(core); + } else { + dev_err(core->dev, "queue full\n"); + return -ENODATA; + } + + return 0; +} + +int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt, u32 pkt_size) +{ + int ret; + + mutex_lock(&core->lock); + ret = iris_hfi_queue_cmd_write_locked(core, pkt, pkt_size); + if (ret) + dev_err(core->dev, "iris_hfi_queue_cmd_write_locked failed with %d\n", ret); + + mutex_unlock(&core->lock); + + return ret; +} + +int iris_hfi_queue_msg_read(struct iris_core *core, void *pkt) +{ + struct iris_iface_q_info *q_info; + int ret = 0; + + mutex_lock(&core->lock); + if (core->state != IRIS_CORE_INIT) { + ret = -EINVAL; + goto unlock; + } + + q_info = &core->message_queue; + if (iris_hfi_queue_read(q_info, pkt)) { + ret = -ENODATA; + goto unlock; + } + +unlock: + mutex_unlock(&core->lock); + + return ret; +} + +int iris_hfi_queue_dbg_read(struct iris_core *core, void *pkt) +{ + struct iris_iface_q_info *q_info; + int ret = 0; + + mutex_lock(&core->lock); + if (core->state != IRIS_CORE_INIT) { + ret = -EINVAL; + goto unlock; + } + + q_info = &core->debug_queue; + if (!q_info || !q_info->kernel_vaddr || !pkt) { + dev_err(core->dev, "cannot read from shared debug queue\n"); + ret = -ENODATA; + goto unlock; + } + + if (iris_hfi_queue_read(q_info, pkt)) { + ret = -ENODATA; + goto unlock; + } + +unlock: + mutex_unlock(&core->lock); + + return ret; +} static void iris_hfi_queue_set_header(struct iris_core *core, u32 queue_id, struct iris_iface_q_info *iface_q) diff --git a/drivers/media/platform/qcom/iris/iris_hfi_queue.h b/drivers/media/platform/qcom/iris/iris_hfi_queue.h index 54994bb776f1..1b728ebddd1d 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_queue.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_queue.h @@ -172,4 +172,9 @@ struct iris_iface_q_info { int iris_hfi_queues_init(struct iris_core *core); void iris_hfi_queues_deinit(struct iris_core *core); +int iris_hfi_queue_cmd_write_locked(struct iris_core *core, void *pkt, u32 pkt_size); +int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt, u32 pkt_size); +int iris_hfi_queue_msg_read(struct iris_core *core, void *pkt); +int iris_hfi_queue_dbg_read(struct iris_core *core, void *pkt); + #endif diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h index 47fdebd8135c..4577977f9f8c 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_common.h +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h @@ -7,6 +7,7 @@ #define _IRIS_PLATFORM_COMMON_H_ #define IRIS_PAS_ID 9 +#define HW_RESPONSE_TIMEOUT_VALUE (1000) /* milliseconds */ extern struct iris_platform_data sm8550_data; extern struct iris_platform_data sm8250_data; @@ -29,7 +30,19 @@ struct tz_cp_config { u32 cp_nonpixel_size; }; +struct ubwc_config_data { + u32 max_channels; + u32 mal_length; + u32 highest_bank_bit; + u32 bank_swzl_level; + u32 bank_swz2_level; + u32 bank_swz3_level; + u32 bank_spreading; +}; + struct iris_platform_data { + void (*init_hfi_command_ops)(struct iris_core *core); + void (*init_hfi_response_ops)(struct iris_core *core); struct iris_inst *(*get_instance)(void); const struct icc_info *icc_tbl; unsigned int icc_tbl_size; @@ -46,6 +59,8 @@ struct iris_platform_data { u32 pas_id; struct tz_cp_config *tz_cp_config_data; u32 core_arch; + u32 hw_response_timeout; + struct ubwc_config_data *ubwc_config; }; #endif diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c index 36bcb1b851ed..b83665a9c5a2 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c @@ -34,6 +34,8 @@ static struct tz_cp_config tz_cp_config_sm8250 = { struct iris_platform_data sm8250_data = { .get_instance = iris_hfi_gen1_get_instance, + .init_hfi_command_ops = &iris_hfi_gen1_command_ops_init, + .init_hfi_response_ops = iris_hfi_gen1_response_ops_init, .icc_tbl = sm8250_icc_table, .icc_tbl_size = ARRAY_SIZE(sm8250_icc_table), .clk_rst_tbl = sm8250_clk_reset_table, @@ -48,4 +50,5 @@ struct iris_platform_data sm8250_data = { .fwname = "qcom/vpu/vpu20_p4.mbn", .pas_id = IRIS_PAS_ID, .tz_cp_config_data = &tz_cp_config_sm8250, + .hw_response_timeout = HW_RESPONSE_TIMEOUT_VALUE, }; diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c index a559e095fefc..91bfc0fa0989 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c @@ -27,6 +27,16 @@ static const struct platform_clk_data sm8550_clk_table[] = { {IRIS_HW_CLK, "vcodec0_core" }, }; +static struct ubwc_config_data ubwc_config_sm8550 = { + .max_channels = 8, + .mal_length = 32, + .highest_bank_bit = 16, + .bank_swzl_level = 0, + .bank_swz2_level = 1, + .bank_swz3_level = 1, + .bank_spreading = 1, +}; + static struct tz_cp_config tz_cp_config_sm8550 = { .cp_start = 0, .cp_size = 0x25800000, @@ -36,6 +46,8 @@ static struct tz_cp_config tz_cp_config_sm8550 = { struct iris_platform_data sm8550_data = { .get_instance = iris_hfi_gen2_get_instance, + .init_hfi_command_ops = iris_hfi_gen2_command_ops_init, + .init_hfi_response_ops = iris_hfi_gen2_response_ops_init, .icc_tbl = sm8550_icc_table, .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table), .clk_rst_tbl = sm8550_clk_reset_table, @@ -51,4 +63,6 @@ struct iris_platform_data sm8550_data = { .pas_id = IRIS_PAS_ID, .tz_cp_config_data = &tz_cp_config_sm8550, .core_arch = VIDEO_ARCH_LX, + .hw_response_timeout = HW_RESPONSE_TIMEOUT_VALUE, + .ubwc_config = &ubwc_config_sm8550, }; diff --git a/drivers/media/platform/qcom/iris/iris_probe.c b/drivers/media/platform/qcom/iris/iris_probe.c index 64bac287d3a8..ecf1a50be63b 100644 --- a/drivers/media/platform/qcom/iris/iris_probe.c +++ b/drivers/media/platform/qcom/iris/iris_probe.c @@ -8,6 +8,21 @@ #include "iris_core.h" #include "iris_vidc.h" +static inline int iris_init_isr(struct iris_core *core) +{ + int ret; + + ret = devm_request_threaded_irq(core->dev, core->irq, iris_hfi_isr, + iris_hfi_isr_handler, IRQF_TRIGGER_HIGH, "iris", core); + if (ret) { + dev_err(core->dev, "failed to allocate irq\n"); + return ret; + } + disable_irq_nosync(core->irq); + + return ret; +} + static int iris_register_video_device(struct iris_core *core) { struct video_device *vdev; @@ -57,6 +72,15 @@ static void iris_remove(struct platform_device *pdev) core->state = IRIS_CORE_DEINIT; } +static void iris_sys_error_handler(struct work_struct *work) +{ + struct iris_core *core = + container_of(work, struct iris_core, sys_error_handler.work); + + iris_core_deinit(core); + iris_core_init(core); +} + static int iris_probe(struct platform_device *pdev) { struct device *dev = &pdev->dev; @@ -71,6 +95,13 @@ static int iris_probe(struct platform_device *pdev) core->state = IRIS_CORE_DEINIT; mutex_init(&core->lock); + init_completion(&core->core_init_done); + + core->response_packet = devm_kzalloc(core->dev, IFACEQ_CORE_PKT_SIZE, GFP_KERNEL); + if (!core->response_packet) + return -ENOMEM; + + INIT_DELAYED_WORK(&core->sys_error_handler, iris_sys_error_handler); core->reg_base = devm_platform_ioremap_resource(pdev, 0); if (IS_ERR(core->reg_base)) @@ -80,6 +111,12 @@ static int iris_probe(struct platform_device *pdev) if (core->irq < 0) return core->irq; + ret = iris_init_isr(core); + if (ret) { + dev_err_probe(core->dev, ret, "Failed to init isr\n"); + return ret; + } + core->iris_platform_data = of_device_get_match_data(core->dev); if (!core->iris_platform_data) { ret = -ENODEV; @@ -88,6 +125,9 @@ static int iris_probe(struct platform_device *pdev) } iris_init_ops(core); + core->iris_platform_data->init_hfi_command_ops(core); + core->iris_platform_data->init_hfi_response_ops(core); + ret = iris_init_resources(core); if (ret) { dev_err_probe(core->dev, ret, "init resource failed\n"); diff --git a/drivers/media/platform/qcom/iris/iris_state.c b/drivers/media/platform/qcom/iris/iris_state.c new file mode 100644 index 000000000000..043aa0ee80b6 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_state.c @@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_core.h" +#include "iris_state.h" + +void iris_change_core_state(struct iris_core *core, + enum iris_core_state request_state) +{ + mutex_lock(&core->lock); + core->state = request_state; + mutex_unlock(&core->lock); +} diff --git a/drivers/media/platform/qcom/iris/iris_state.h b/drivers/media/platform/qcom/iris/iris_state.h index a0bfee98578a..8bf22f839311 100644 --- a/drivers/media/platform/qcom/iris/iris_state.h +++ b/drivers/media/platform/qcom/iris/iris_state.h @@ -6,6 +6,8 @@ #ifndef _IRIS_STATE_H_ #define _IRIS_STATE_H_ +struct iris_core; + /** * enum iris_core_state * @@ -38,4 +40,6 @@ enum iris_core_state { IRIS_CORE_ERROR, }; +void iris_change_core_state(struct iris_core *core, + enum iris_core_state request_state); #endif diff --git a/drivers/media/platform/qcom/iris/iris_vpu_common.c b/drivers/media/platform/qcom/iris/iris_vpu_common.c index df87b1b719a9..5930cf105a87 100644 --- a/drivers/media/platform/qcom/iris/iris_vpu_common.c +++ b/drivers/media/platform/qcom/iris/iris_vpu_common.c @@ -11,6 +11,9 @@ #define CPU_BASE_OFFS 0x000A0000 #define CPU_CS_BASE_OFFS (CPU_BASE_OFFS) +#define CPU_IC_BASE_OFFS (CPU_BASE_OFFS) + +#define CPU_CS_A2HSOFTINTCLR (CPU_CS_BASE_OFFS + 0x1C) #define CTRL_INIT (CPU_CS_BASE_OFFS + 0x48) #define CTRL_STATUS (CPU_CS_BASE_OFFS + 0x4C) @@ -27,6 +30,15 @@ #define CPU_CS_H2XSOFTINTEN (CPU_CS_BASE_OFFS + 0x148) #define CPU_CS_X2RPMH (CPU_CS_BASE_OFFS + 0x168) +#define CPU_IC_SOFTINT (CPU_IC_BASE_OFFS + 0x150) +#define CPU_IC_SOFTINT_H2A_SHFT 0x0 + +#define WRAPPER_BASE_OFFS 0x000B0000 +#define WRAPPER_INTR_STATUS (WRAPPER_BASE_OFFS + 0x0C) +#define WRAPPER_INTR_STATUS_A2HWD_BMSK 0x8 +#define WRAPPER_INTR_STATUS_A2H_BMSK 0x4 +#define CTRL_INIT_IDLE_MSG_BMSK 0x40000000 + static void iris_vpu_setup_ucregion_memory_map(struct iris_core *core) { u32 queue_size, value; @@ -85,3 +97,33 @@ int iris_vpu_boot_firmware(struct iris_core *core) return 0; } + +void iris_vpu_raise_interrupt(struct iris_core *core) +{ + writel(1 << CPU_IC_SOFTINT_H2A_SHFT, core->reg_base + CPU_IC_SOFTINT); +} + +void iris_vpu_clear_interrupt(struct iris_core *core) +{ + u32 intr_status = 0, mask = 0; + + intr_status = readl(core->reg_base + WRAPPER_INTR_STATUS); + mask = (WRAPPER_INTR_STATUS_A2H_BMSK | + WRAPPER_INTR_STATUS_A2HWD_BMSK | + CTRL_INIT_IDLE_MSG_BMSK); + + if (intr_status & mask) + core->intr_status |= intr_status; + + writel(1, core->reg_base + CPU_CS_A2HSOFTINTCLR); +} + +int iris_vpu_watchdog(struct iris_core *core, u32 intr_status) +{ + if (intr_status & WRAPPER_INTR_STATUS_A2HWD_BMSK) { + dev_err(core->dev, "received interrupt\n"); + return -ETIME; + } + + return 0; +} diff --git a/drivers/media/platform/qcom/iris/iris_vpu_common.h b/drivers/media/platform/qcom/iris/iris_vpu_common.h index d9b8df6e3f80..706b207bc295 100644 --- a/drivers/media/platform/qcom/iris/iris_vpu_common.h +++ b/drivers/media/platform/qcom/iris/iris_vpu_common.h @@ -9,5 +9,8 @@ struct iris_core; int iris_vpu_boot_firmware(struct iris_core *core); +void iris_vpu_raise_interrupt(struct iris_core *core); +void iris_vpu_clear_interrupt(struct iris_core *core); +int iris_vpu_watchdog(struct iris_core *core, u32 intr_status); #endif From patchwork Tue Aug 27 10:05:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 822726 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C09519E81F; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; cv=none; b=MhrHZxTYhKApLKggUMUAaqeG3MeUi64p8ud90iMTjExK0XGU0kTEU84XDcjMyPe0CMzWb5x7ZPxk89Swe5AB0wiCmdjGnND5XbP9ajhJftICkYGl89bpZtwjJBNQvVSIelt2y/6C7lC0nbW/UzFSi63Lpp5SazChS46Kzq5ORkM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; c=relaxed/simple; bh=ZuAPvKSbqYfqwq91wQ1If8zgBTs6irQujGVfq2+jRLc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Hbra44rwi1iU6W1i35mLlnyAdCz2sJi3gB84dHpUTgEmBWonYB9qgAXDdjF4UkKFX0PoJ0c+yxRxMwvHNXY/OXKehyjTP+HCwYzPS/RCOjLRhChwtN0+hnTyONZNe2m8UjCjuwt270bnUx5wQQiCoHWK7k4OJVKLDW3uCBQO1zE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Vo5ePoWo; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Vo5ePoWo" Received: by smtp.kernel.org (Postfix) with ESMTPS id 155EAC4DDED; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753195; bh=ZuAPvKSbqYfqwq91wQ1If8zgBTs6irQujGVfq2+jRLc=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=Vo5ePoWoucOPPYKTc/0fdDgVXBB1NS6V9Ewg9JdLp1LAfmYqwfCs4Anx9F0V5iLUj bJ84eYXj4NV8D1XVSevOSsmwuYHek5CYV4LniuDykH7J8CSVWHQpgyvi1vC2Ifd8gA KKlcxyaIVAsz2Py298oht4+Gqw1hLEBRwRkrOc0DITaoZ/8ExcU1/Mf07HIafPSI7S ClpDF9wdlH8u0nSxcx++tcPn/pgpWKibUYK65Cxr0sWw/BYz9YJmnQT+Piav1Eubz5 vguv2/PCFMJTK3/t9LrlXnpgTFVui40GONtb/XP0jG3kIMs3DQP3Za0D2RgIUV2tAs uBGkTzziVMppw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 05D94C54732; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:35 +0530 Subject: [PATCH v3 10/29] media: iris: implement power management Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-10-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=44362; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=db5KZurs+6DzDTrjvTJKSa+j0rf2+ys3m/s7/zLCrGo=; b=lSLZO7dnE6PNLzt2WnwKKlhQ7+x0Y32rNSRtIYXrca/UpwvOqVQSqDWrDXO2UgxtTOGxJTH0y h7Bsf2HSN++DLTl0bUndtxUzrzsftN3VqwtzpZhp1jBqbhgpARa2XUU X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal Implement runtime power management for iris including platform specific power on/off sequence. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/Makefile | 3 + drivers/media/platform/qcom/iris/iris_core.c | 16 +- drivers/media/platform/qcom/iris/iris_core.h | 5 + drivers/media/platform/qcom/iris/iris_firmware.c | 5 + drivers/media/platform/qcom/iris/iris_firmware.h | 1 + drivers/media/platform/qcom/iris/iris_hfi_common.c | 72 +++++- drivers/media/platform/qcom/iris/iris_hfi_common.h | 3 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 11 + .../platform/qcom/iris/iris_hfi_gen1_defines.h | 5 + .../platform/qcom/iris/iris_hfi_gen2_command.c | 20 ++ .../platform/qcom/iris/iris_hfi_gen2_defines.h | 1 + .../platform/qcom/iris/iris_hfi_gen2_packet.c | 13 ++ .../platform/qcom/iris/iris_hfi_gen2_packet.h | 1 + drivers/media/platform/qcom/iris/iris_hfi_queue.c | 20 +- .../platform/qcom/iris/iris_platform_common.h | 14 ++ .../platform/qcom/iris/iris_platform_sm8250.c | 9 + .../platform/qcom/iris/iris_platform_sm8550.c | 9 + drivers/media/platform/qcom/iris/iris_power.c | 35 +++ drivers/media/platform/qcom/iris/iris_power.h | 14 ++ drivers/media/platform/qcom/iris/iris_probe.c | 66 +++++- drivers/media/platform/qcom/iris/iris_resources.c | 167 ++++++++++++++ drivers/media/platform/qcom/iris/iris_resources.h | 8 + drivers/media/platform/qcom/iris/iris_vidc.c | 8 + drivers/media/platform/qcom/iris/iris_vpu2.c | 11 + drivers/media/platform/qcom/iris/iris_vpu3.c | 79 +++++++ drivers/media/platform/qcom/iris/iris_vpu_common.c | 250 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_vpu_common.h | 11 + .../platform/qcom/iris/iris_vpu_register_defines.h | 17 ++ 28 files changed, 862 insertions(+), 12 deletions(-) diff --git a/drivers/media/platform/qcom/iris/Makefile b/drivers/media/platform/qcom/iris/Makefile index d1f0b933df3d..b0447af45891 100644 --- a/drivers/media/platform/qcom/iris/Makefile +++ b/drivers/media/platform/qcom/iris/Makefile @@ -9,10 +9,13 @@ iris-objs += iris_core.o \ iris_hfi_queue.o \ iris_platform_sm8250.o \ iris_platform_sm8550.o \ + iris_power.o \ iris_probe.o \ iris_resources.o \ iris_state.o \ iris_vidc.o \ + iris_vpu2.o \ + iris_vpu3.o \ iris_vpu_common.o \ obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/iris/iris_core.c b/drivers/media/platform/qcom/iris/iris_core.c index 92458d7f1e36..34847fb36aa0 100644 --- a/drivers/media/platform/qcom/iris/iris_core.c +++ b/drivers/media/platform/qcom/iris/iris_core.c @@ -3,18 +3,26 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include + #include "iris_core.h" #include "iris_firmware.h" +#include "iris_power.h" #include "iris_state.h" #include "iris_vpu_common.h" void iris_core_deinit(struct iris_core *core) { + pm_runtime_resume_and_get(core->dev); + mutex_lock(&core->lock); iris_fw_unload(core); + iris_power_off(core); iris_hfi_queues_deinit(core); core->state = IRIS_CORE_DEINIT; mutex_unlock(&core->lock); + + pm_runtime_put_sync(core->dev); } static int iris_wait_for_system_response(struct iris_core *core) @@ -56,10 +64,14 @@ int iris_core_init(struct iris_core *core) if (ret) goto error; - ret = iris_fw_load(core); + ret = iris_power_on(core); if (ret) goto error_queue_deinit; + ret = iris_fw_load(core); + if (ret) + goto error_power_off; + ret = iris_vpu_boot_firmware(core); if (ret) goto error_unload_fw; @@ -74,6 +86,8 @@ int iris_core_init(struct iris_core *core) error_unload_fw: iris_fw_unload(core); +error_power_off: + iris_power_off(core); error_queue_deinit: iris_hfi_queues_deinit(core); error: diff --git a/drivers/media/platform/qcom/iris/iris_core.h b/drivers/media/platform/qcom/iris/iris_core.h index 409f9822807d..09a6904c7bb1 100644 --- a/drivers/media/platform/qcom/iris/iris_core.h +++ b/drivers/media/platform/qcom/iris/iris_core.h @@ -7,6 +7,7 @@ #define _IRIS_CORE_H_ #include +#include #include #include "iris_hfi_common.h" @@ -47,6 +48,8 @@ * @response_packet: a pointer to response packet from fw to driver * @header_id: id of packet header * @packet_id: id of packet + * @power_enabled: a boolean to check if power is on or off + * @power: a structure for clock and bw information * @hfi_ops: iris hfi command ops * @hfi_response_ops: iris hfi response ops * @core_init_done: structure of signal completion for system response @@ -81,6 +84,8 @@ struct iris_core { u8 *response_packet; u32 header_id; u32 packet_id; + bool power_enabled; + struct iris_core_power power; const struct iris_hfi_command_ops *hfi_ops; const struct iris_hfi_response_ops *hfi_response_ops; struct completion core_init_done; diff --git a/drivers/media/platform/qcom/iris/iris_firmware.c b/drivers/media/platform/qcom/iris/iris_firmware.c index 55bbcc798f4c..b6c779ec13bb 100644 --- a/drivers/media/platform/qcom/iris/iris_firmware.c +++ b/drivers/media/platform/qcom/iris/iris_firmware.c @@ -144,3 +144,8 @@ int iris_fw_unload(struct iris_core *core) return ret; } + +int iris_set_hw_state(struct iris_core *core, bool resume) +{ + return qcom_scm_set_remote_state(resume, 0); +} diff --git a/drivers/media/platform/qcom/iris/iris_firmware.h b/drivers/media/platform/qcom/iris/iris_firmware.h index 8d4f6b7f75c5..1550916c92a1 100644 --- a/drivers/media/platform/qcom/iris/iris_firmware.h +++ b/drivers/media/platform/qcom/iris/iris_firmware.h @@ -10,5 +10,6 @@ struct iris_core; int iris_fw_load(struct iris_core *core); int iris_fw_unload(struct iris_core *core); +int iris_set_hw_state(struct iris_core *core, bool resume); #endif diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.c b/drivers/media/platform/qcom/iris/iris_hfi_common.c index a5a28029d8d1..bf9ec678ff90 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_common.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_common.c @@ -3,8 +3,12 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include + +#include "iris_firmware.h" #include "iris_core.h" #include "iris_hfi_common.h" +#include "iris_power.h" #include "iris_vpu_common.h" int iris_hfi_core_init(struct iris_core *core) @@ -42,7 +46,7 @@ irqreturn_t iris_hfi_isr_handler(int irq, void *data) mutex_unlock(&core->lock); goto exit; } - + pm_runtime_mark_last_busy(core->dev); iris_vpu_clear_interrupt(core); mutex_unlock(&core->lock); @@ -54,3 +58,69 @@ irqreturn_t iris_hfi_isr_handler(int irq, void *data) return IRQ_HANDLED; } + +int iris_hfi_pm_suspend(struct iris_core *core) +{ + int ret; + + if (!mutex_is_locked(&core->lock)) + return -EINVAL; + + if (core->state != IRIS_CORE_INIT) + return -EINVAL; + + if (!core->power_enabled) { + dev_err(core->dev, "power not enabled\n"); + return 0; + } + + ret = iris_vpu_prepare_pc(core); + if (ret) { + dev_err(core->dev, "prepare pc ret %d\n", ret); + pm_runtime_mark_last_busy(core->dev); + return -EAGAIN; + } + + ret = iris_set_hw_state(core, false); + if (ret) + return ret; + + iris_power_off(core); + + return 0; +} + +int iris_hfi_pm_resume(struct iris_core *core) +{ + const struct iris_hfi_command_ops *ops; + int ret; + + ops = core->hfi_ops; + + ret = iris_power_on(core); + if (ret) + goto error; + + ret = iris_set_hw_state(core, true); + if (ret) + goto err_power_off; + + ret = iris_vpu_boot_firmware(core); + if (ret) + goto err_suspend_hw; + + ret = ops->sys_interframe_powercollapse(core); + if (ret) + goto err_suspend_hw; + + return 0; + +err_suspend_hw: + iris_set_hw_state(core, false); +err_power_off: + iris_power_off(core); +error: + dev_err(core->dev, "failed to resume\n"); + + return -EBUSY; +} diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.h b/drivers/media/platform/qcom/iris/iris_hfi_common.h index c3d5b899cf60..e050b1ae90fe 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_common.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_common.h @@ -46,6 +46,7 @@ struct iris_hfi_command_ops { int (*sys_init)(struct iris_core *core); int (*sys_image_version)(struct iris_core *core); int (*sys_interframe_powercollapse)(struct iris_core *core); + int (*sys_pc_prep)(struct iris_core *core); }; struct iris_hfi_response_ops { @@ -53,6 +54,8 @@ struct iris_hfi_response_ops { }; int iris_hfi_core_init(struct iris_core *core); +int iris_hfi_pm_suspend(struct iris_core *core); +int iris_hfi_pm_resume(struct iris_core *core); irqreturn_t iris_hfi_isr(int irq, void *data); irqreturn_t iris_hfi_isr_handler(int irq, void *data); diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c index 8f045ef56163..e778ae33b953 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c @@ -56,10 +56,21 @@ static int iris_hfi_gen1_sys_interframe_powercollapse(struct iris_core *core) return ret; } +static int iris_hfi_gen1_sys_pc_prep(struct iris_core *core) +{ + struct hfi_sys_pc_prep_pkt pkt; + + pkt.hdr.size = sizeof(struct hfi_sys_pc_prep_pkt); + pkt.hdr.pkt_type = HFI_CMD_SYS_PC_PREP; + + return iris_hfi_queue_cmd_write_locked(core, &pkt, pkt.hdr.size); +} + static const struct iris_hfi_command_ops iris_hfi_gen1_command_ops = { .sys_init = iris_hfi_gen1_sys_init, .sys_image_version = iris_hfi_gen1_sys_image_version, .sys_interframe_powercollapse = iris_hfi_gen1_sys_interframe_powercollapse, + .sys_pc_prep = iris_hfi_gen1_sys_pc_prep, }; void iris_hfi_gen1_command_ops_init(struct iris_core *core) diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h index 5c07d6a29863..991d5a5dc792 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h @@ -12,6 +12,7 @@ #define HFI_ERR_NONE 0x0 #define HFI_CMD_SYS_INIT 0x10001 +#define HFI_CMD_SYS_PC_PREP 0x10002 #define HFI_CMD_SYS_SET_PROPERTY 0x10005 #define HFI_CMD_SYS_GET_PROPERTY 0x10006 @@ -48,6 +49,10 @@ struct hfi_sys_get_property_pkt { u32 data; }; +struct hfi_sys_pc_prep_pkt { + struct hfi_pkt_hdr hdr; +}; + struct hfi_msg_event_notify_pkt { struct hfi_pkt_hdr hdr; u32 event_id; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c index 807266858d93..0871c0bdea90 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c @@ -66,10 +66,30 @@ static int iris_hfi_gen2_sys_interframe_powercollapse(struct iris_core *core) return ret; } +static int iris_hfi_gen2_sys_pc_prep(struct iris_core *core) +{ + struct iris_hfi_header *hdr; + u32 packet_size; + int ret; + + packet_size = sizeof(*hdr) + sizeof(struct iris_hfi_packet); + hdr = kzalloc(packet_size, GFP_KERNEL); + if (!hdr) + return -ENOMEM; + + iris_hfi_gen2_packet_sys_pc_prep(core, hdr); + ret = iris_hfi_queue_cmd_write_locked(core, hdr, hdr->size); + + kfree(hdr); + + return ret; +} + static const struct iris_hfi_command_ops iris_hfi_gen2_command_ops = { .sys_init = iris_hfi_gen2_sys_init, .sys_image_version = iris_hfi_gen2_sys_image_version, .sys_interframe_powercollapse = iris_hfi_gen2_sys_interframe_powercollapse, + .sys_pc_prep = iris_hfi_gen2_sys_pc_prep, }; void iris_hfi_gen2_command_ops_init(struct iris_core *core) diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h index 3e3e4ddfe21f..4104479c7251 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h @@ -12,6 +12,7 @@ #define HFI_CMD_BEGIN 0x01000000 #define HFI_CMD_INIT 0x01000001 +#define HFI_CMD_POWER_COLLAPSE 0x01000002 #define HFI_CMD_END 0x01FFFFFF #define HFI_PROP_BEGIN 0x03000000 diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c index 8266eae5ff94..9ea26328a300 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c @@ -162,3 +162,16 @@ void iris_hfi_gen2_packet_sys_interframe_powercollapse(struct iris_core *core, &payload, sizeof(u32)); } + +void iris_hfi_gen2_packet_sys_pc_prep(struct iris_core *core, struct iris_hfi_header *hdr) +{ + iris_hfi_gen2_create_header(hdr, 0 /*session_id*/, core->header_id++); + + iris_hfi_gen2_create_packet(hdr, + HFI_CMD_POWER_COLLAPSE, + HFI_HOST_FLAGS_NONE, + HFI_PAYLOAD_NONE, + HFI_PORT_NONE, + core->packet_id++, + NULL, 0); +} diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h index eba109efeb76..163295783b7d 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h @@ -65,5 +65,6 @@ void iris_hfi_gen2_packet_sys_init(struct iris_core *core, struct iris_hfi_heade void iris_hfi_gen2_packet_image_version(struct iris_core *core, struct iris_hfi_header *hdr); void iris_hfi_gen2_packet_sys_interframe_powercollapse(struct iris_core *core, struct iris_hfi_header *hdr); +void iris_hfi_gen2_packet_sys_pc_prep(struct iris_core *core, struct iris_hfi_header *hdr); #endif diff --git a/drivers/media/platform/qcom/iris/iris_hfi_queue.c b/drivers/media/platform/qcom/iris/iris_hfi_queue.c index b24d4640fea9..3a511d5e5cfc 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_queue.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_queue.c @@ -3,6 +3,8 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include + #include "iris_core.h" #include "iris_hfi_queue.h" #include "iris_vpu_common.h" @@ -145,13 +147,27 @@ int iris_hfi_queue_cmd_write(struct iris_core *core, void *pkt, u32 pkt_size) { int ret; + ret = pm_runtime_resume_and_get(core->dev); + if (ret < 0) + goto exit; + mutex_lock(&core->lock); ret = iris_hfi_queue_cmd_write_locked(core, pkt, pkt_size); - if (ret) + if (ret) { dev_err(core->dev, "iris_hfi_queue_cmd_write_locked failed with %d\n", ret); - + mutex_unlock(&core->lock); + goto exit; + } mutex_unlock(&core->lock); + pm_runtime_mark_last_busy(core->dev); + pm_runtime_put_autosuspend(core->dev); + + return 0; + +exit: + pm_runtime_put_sync(core->dev); + return ret; } diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h index 4577977f9f8c..899a696a931d 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_common.h +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h @@ -8,6 +8,7 @@ #define IRIS_PAS_ID 9 #define HW_RESPONSE_TIMEOUT_VALUE (1000) /* milliseconds */ +#define AUTOSUSPEND_DELAY_VALUE (HW_RESPONSE_TIMEOUT_VALUE + 500) /* milliseconds */ extern struct iris_platform_data sm8550_data; extern struct iris_platform_data sm8250_data; @@ -40,10 +41,22 @@ struct ubwc_config_data { u32 bank_spreading; }; +struct iris_core_power { + u64 clk_freq; + u64 icc_bw; +}; + +enum platform_pm_domain_type { + IRIS_CTRL_POWER_DOMAIN, + IRIS_HW_POWER_DOMAIN, +}; + struct iris_platform_data { void (*init_hfi_command_ops)(struct iris_core *core); void (*init_hfi_response_ops)(struct iris_core *core); struct iris_inst *(*get_instance)(void); + const struct vpu_ops *vpu_ops; + void (*set_preset_registers)(struct iris_core *core); const struct icc_info *icc_tbl; unsigned int icc_tbl_size; const char * const *pmdomain_tbl; @@ -61,6 +74,7 @@ struct iris_platform_data { u32 core_arch; u32 hw_response_timeout; struct ubwc_config_data *ubwc_config; + u32 num_vpp_pipe; }; #endif diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c index b83665a9c5a2..1adbafa373a5 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c @@ -7,6 +7,12 @@ #include "iris_platform_common.h" #include "iris_resources.h" #include "iris_hfi_gen1.h" +#include "iris_vpu_common.h" + +static void iris_set_sm8250_preset_registers(struct iris_core *core) +{ + writel(0x0, core->reg_base + 0xB0088); +} static const struct icc_info sm8250_icc_table[] = { { "cpu-cfg", 1000, 1000 }, @@ -36,6 +42,8 @@ struct iris_platform_data sm8250_data = { .get_instance = iris_hfi_gen1_get_instance, .init_hfi_command_ops = &iris_hfi_gen1_command_ops_init, .init_hfi_response_ops = iris_hfi_gen1_response_ops_init, + .vpu_ops = &iris_vpu2_ops, + .set_preset_registers = iris_set_sm8250_preset_registers, .icc_tbl = sm8250_icc_table, .icc_tbl_size = ARRAY_SIZE(sm8250_icc_table), .clk_rst_tbl = sm8250_clk_reset_table, @@ -51,4 +59,5 @@ struct iris_platform_data sm8250_data = { .pas_id = IRIS_PAS_ID, .tz_cp_config_data = &tz_cp_config_sm8250, .hw_response_timeout = HW_RESPONSE_TIMEOUT_VALUE, + .num_vpp_pipe = 4, }; diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c index 91bfc0fa0989..950ccdccde31 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c @@ -7,9 +7,15 @@ #include "iris_hfi_gen2.h" #include "iris_platform_common.h" #include "iris_resources.h" +#include "iris_vpu_common.h" #define VIDEO_ARCH_LX 1 +static void iris_set_sm8550_preset_registers(struct iris_core *core) +{ + writel(0x0, core->reg_base + 0xB0088); +} + static const struct icc_info sm8550_icc_table[] = { { "cpu-cfg", 1000, 1000 }, { "video-mem", 1000, 15000000 }, @@ -48,6 +54,8 @@ struct iris_platform_data sm8550_data = { .get_instance = iris_hfi_gen2_get_instance, .init_hfi_command_ops = iris_hfi_gen2_command_ops_init, .init_hfi_response_ops = iris_hfi_gen2_response_ops_init, + .vpu_ops = &iris_vpu3_ops, + .set_preset_registers = iris_set_sm8550_preset_registers, .icc_tbl = sm8550_icc_table, .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table), .clk_rst_tbl = sm8550_clk_reset_table, @@ -65,4 +73,5 @@ struct iris_platform_data sm8550_data = { .core_arch = VIDEO_ARCH_LX, .hw_response_timeout = HW_RESPONSE_TIMEOUT_VALUE, .ubwc_config = &ubwc_config_sm8550, + .num_vpp_pipe = 4, }; diff --git a/drivers/media/platform/qcom/iris/iris_power.c b/drivers/media/platform/qcom/iris/iris_power.c new file mode 100644 index 000000000000..e697c27c8a8a --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_power.c @@ -0,0 +1,35 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_core.h" +#include "iris_power.h" +#include "iris_vpu_common.h" + +void iris_power_off(struct iris_core *core) +{ + if (!core->power_enabled) + return; + + iris_vpu_power_off(core); + core->power_enabled = false; +} + +int iris_power_on(struct iris_core *core) +{ + int ret; + + if (core->power_enabled) + return 0; + + ret = iris_vpu_power_on(core); + if (ret) { + dev_err(core->dev, "failed to power on, err: %d\n", ret); + return ret; + } + + core->power_enabled = true; + + return ret; +} diff --git a/drivers/media/platform/qcom/iris/iris_power.h b/drivers/media/platform/qcom/iris/iris_power.h new file mode 100644 index 000000000000..ff9b6be3b086 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_power.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_POWER_H_ +#define _IRIS_POWER_H_ + +struct iris_core; + +int iris_power_on(struct iris_core *core); +void iris_power_off(struct iris_core *core); + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_probe.c b/drivers/media/platform/qcom/iris/iris_probe.c index ecf1a50be63b..3222e9116551 100644 --- a/drivers/media/platform/qcom/iris/iris_probe.c +++ b/drivers/media/platform/qcom/iris/iris_probe.c @@ -4,6 +4,7 @@ */ #include +#include #include "iris_core.h" #include "iris_vidc.h" @@ -111,17 +112,25 @@ static int iris_probe(struct platform_device *pdev) if (core->irq < 0) return core->irq; + pm_runtime_set_autosuspend_delay(core->dev, AUTOSUSPEND_DELAY_VALUE); + pm_runtime_use_autosuspend(core->dev); + ret = devm_pm_runtime_enable(core->dev); + if (ret) { + dev_err(core->dev, "failed to enable runtime pm\n"); + goto err_runtime_disable; + } + ret = iris_init_isr(core); if (ret) { dev_err_probe(core->dev, ret, "Failed to init isr\n"); - return ret; + goto err_runtime_disable; } core->iris_platform_data = of_device_get_match_data(core->dev); if (!core->iris_platform_data) { ret = -ENODEV; dev_err_probe(core->dev, ret, "init platform failed\n"); - return ret; + goto err_runtime_disable; } iris_init_ops(core); @@ -131,12 +140,12 @@ static int iris_probe(struct platform_device *pdev) ret = iris_init_resources(core); if (ret) { dev_err_probe(core->dev, ret, "init resource failed\n"); - return ret; + goto err_runtime_disable; } ret = v4l2_device_register(dev, &core->v4l2_dev); if (ret) - return ret; + goto err_runtime_disable; ret = iris_register_video_device(core); if (ret) @@ -159,10 +168,58 @@ static int iris_probe(struct platform_device *pdev) video_unregister_device(core->vdev_dec); err_v4l2_unreg: v4l2_device_unregister(&core->v4l2_dev); +err_runtime_disable: + pm_runtime_set_suspended(core->dev); + + return ret; +} + +static int iris_pm_suspend(struct device *dev) +{ + struct iris_core *core; + int ret; + + if (!dev || !dev->driver) + return 0; + + core = dev_get_drvdata(dev); + + mutex_lock(&core->lock); + ret = iris_hfi_pm_suspend(core); + mutex_unlock(&core->lock); return ret; } +static int iris_pm_resume(struct device *dev) +{ + struct iris_core *core; + int ret = 0; + + if (!dev || !dev->driver) + return 0; + + core = dev_get_drvdata(dev); + + mutex_lock(&core->lock); + if (core->state != IRIS_CORE_INIT) + goto exit; + + ret = iris_hfi_pm_resume(core); + pm_runtime_mark_last_busy(core->dev); + +exit: + mutex_unlock(&core->lock); + + return ret; +} + +static const struct dev_pm_ops iris_pm_ops = { + SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, + pm_runtime_force_resume) + SET_RUNTIME_PM_OPS(iris_pm_suspend, iris_pm_resume, NULL) +}; + static const struct of_device_id iris_dt_match[] = { { .compatible = "qcom,sm8550-iris", @@ -182,6 +239,7 @@ static struct platform_driver qcom_iris_driver = { .driver = { .name = "qcom-iris", .of_match_table = iris_dt_match, + .pm = &iris_pm_ops, }, }; diff --git a/drivers/media/platform/qcom/iris/iris_resources.c b/drivers/media/platform/qcom/iris/iris_resources.c index 57c6f9f3449b..eb3052acd426 100644 --- a/drivers/media/platform/qcom/iris/iris_resources.c +++ b/drivers/media/platform/qcom/iris/iris_resources.c @@ -7,11 +7,14 @@ #include #include #include +#include #include #include "iris_core.h" #include "iris_resources.h" +#define BW_THRESHOLD 50000 + static int iris_init_icc(struct iris_core *core) { const struct icc_info *icc_tbl; @@ -39,6 +42,57 @@ static int iris_init_icc(struct iris_core *core) return ret; } +int iris_set_icc_bw(struct iris_core *core, unsigned long icc_bw) +{ + unsigned long bw_kbps = 0, bw_prev = 0; + const struct icc_info *icc_tbl; + int ret = 0, i; + + icc_tbl = core->iris_platform_data->icc_tbl; + + for (i = 0; i < core->icc_count; i++) { + if (!strcmp(core->icc_tbl[i].name, "video-mem")) { + bw_kbps = icc_bw; + bw_prev = core->power.icc_bw; + + bw_kbps = clamp_t(typeof(bw_kbps), bw_kbps, + icc_tbl[i].bw_min_kbps, icc_tbl[i].bw_max_kbps); + + if (abs(bw_kbps - bw_prev) < BW_THRESHOLD && bw_prev) + return ret; + + core->icc_tbl[i].avg_bw = bw_kbps; + + core->power.icc_bw = bw_kbps; + break; + } + } + + ret = icc_bulk_set_bw(core->icc_count, core->icc_tbl); + if (ret) + dev_err(core->dev, "failed to unset icc bw\n"); + + return ret; +} + +int iris_unset_icc_bw(struct iris_core *core) +{ + int ret, i; + + core->power.icc_bw = 0; + + for (i = 0; i < core->icc_count; i++) { + core->icc_tbl[i].avg_bw = 0; + core->icc_tbl[i].peak_bw = 0; + } + + ret = icc_bulk_set_bw(core->icc_count, core->icc_tbl); + if (ret) + dev_err(core->dev, "failed to unset icc bw\n"); + + return ret; +} + static int iris_pd_get(struct iris_core *core) { int ret; @@ -73,6 +127,19 @@ static int iris_opp_pd_get(struct iris_core *core) return 0; } +int iris_opp_set_rate(struct iris_core *core, u64 freq) +{ + int ret; + + ret = dev_pm_opp_set_rate(core->dev, freq); + if (ret) { + dev_err(core->dev, "failed to set rate\n"); + return ret; + } + + return ret; +} + static int iris_init_power_domains(struct iris_core *core) { const struct platform_clk_data *clk_tbl; @@ -107,6 +174,36 @@ static int iris_init_power_domains(struct iris_core *core) return ret; } +int iris_enable_power_domains(struct iris_core *core, struct device *pd_dev) +{ + int ret; + + ret = iris_opp_set_rate(core, ULONG_MAX); + if (ret) + return ret; + + ret = pm_runtime_get_sync(pd_dev); + if (ret < 0) + return ret; + + return ret; +} + +int iris_disable_power_domains(struct iris_core *core, struct device *pd_dev) +{ + int ret; + + ret = iris_opp_set_rate(core, 0); + if (ret) + return ret; + + ret = pm_runtime_put_sync(pd_dev); + if (ret) + return ret; + + return ret; +} + static int iris_init_clocks(struct iris_core *core) { int ret; @@ -122,6 +219,62 @@ static int iris_init_clocks(struct iris_core *core) return 0; } +static struct clk *iris_get_clk_by_type(struct iris_core *core, enum platform_clk_type clk_type) +{ + const struct platform_clk_data *clk_tbl; + u32 clk_cnt; + int i, j; + + clk_tbl = core->iris_platform_data->clk_tbl; + clk_cnt = core->iris_platform_data->clk_tbl_size; + + for (i = 0; i < clk_cnt; i++) { + if (clk_tbl[i].clk_type == clk_type) { + for (j = 0; core->clock_tbl && j < core->clk_count; j++) { + if (!strcmp(core->clock_tbl[j].id, clk_tbl[i].clk_name)) + return core->clock_tbl[j].clk; + } + } + } + + return NULL; +} + +int iris_prepare_enable_clock(struct iris_core *core, enum platform_clk_type clk_type) +{ + struct clk *clock; + int ret = 0; + + clock = iris_get_clk_by_type(core, clk_type); + if (!clock) { + dev_err(core->dev, "failed to get clk: %d\n", clk_type); + return -EINVAL; + } + + ret = clk_prepare_enable(clock); + if (ret) { + dev_err(core->dev, "failed to enable clock %d\n", clk_type); + return ret; + } + + return ret; +} + +int iris_disable_unprepare_clock(struct iris_core *core, enum platform_clk_type clk_type) +{ + struct clk *clock; + + clock = iris_get_clk_by_type(core, clk_type); + if (!clock) { + dev_err(core->dev, "failed to get clk: %d\n", clk_type); + return -EINVAL; + } + + clk_disable_unprepare(clock); + + return 0; +} + static int iris_init_resets(struct iris_core *core) { const char * const *rst_tbl; @@ -149,6 +302,20 @@ static int iris_init_resets(struct iris_core *core) return 0; } +int iris_reset_ahb2axi_bridge(struct iris_core *core) +{ + u32 rst_tbl_size; + int ret; + + rst_tbl_size = core->iris_platform_data->clk_rst_tbl_size; + + ret = reset_control_bulk_reset(rst_tbl_size, core->resets); + if (ret) + dev_err(core->dev, "failed to toggle resets: %d\n", ret); + + return ret; +} + int iris_init_resources(struct iris_core *core) { int ret; diff --git a/drivers/media/platform/qcom/iris/iris_resources.h b/drivers/media/platform/qcom/iris/iris_resources.h index b0217399030a..a13d8307cb2c 100644 --- a/drivers/media/platform/qcom/iris/iris_resources.h +++ b/drivers/media/platform/qcom/iris/iris_resources.h @@ -14,6 +14,14 @@ struct icc_info { u32 bw_max_kbps; }; +int iris_enable_power_domains(struct iris_core *core, struct device *pd_dev); +int iris_disable_power_domains(struct iris_core *core, struct device *pd_dev); +int iris_unset_icc_bw(struct iris_core *core); +int iris_set_icc_bw(struct iris_core *core, unsigned long icc_bw); +int iris_reset_ahb2axi_bridge(struct iris_core *core); +int iris_opp_set_rate(struct iris_core *core, u64 freq); +int iris_disable_unprepare_clock(struct iris_core *core, enum platform_clk_type clk_type); +int iris_prepare_enable_clock(struct iris_core *core, enum platform_clk_type clk_type); int iris_init_resources(struct iris_core *core); #endif diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index a5c4e73ae531..bc0a3232214b 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -3,6 +3,7 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include #include #include @@ -96,12 +97,19 @@ int iris_open(struct file *filp) struct iris_inst *inst = NULL; int ret; + ret = pm_runtime_resume_and_get(core->dev); + if (ret < 0) + return ret; + ret = iris_core_init(core); if (ret) { dev_err(core->dev, "core init failed\n"); + pm_runtime_put_sync(core->dev); return ret; } + pm_runtime_put_sync(core->dev); + inst = core->iris_platform_data->get_instance(); if (!inst) return -ENOMEM; diff --git a/drivers/media/platform/qcom/iris/iris_vpu2.c b/drivers/media/platform/qcom/iris/iris_vpu2.c new file mode 100644 index 000000000000..bd8427411576 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_vpu2.c @@ -0,0 +1,11 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_instance.h" +#include "iris_vpu_common.h" + +const struct vpu_ops iris_vpu2_ops = { + .power_off_hw = iris_vpu_power_off_hw, +}; diff --git a/drivers/media/platform/qcom/iris/iris_vpu3.c b/drivers/media/platform/qcom/iris/iris_vpu3.c new file mode 100644 index 000000000000..414b8ec67b33 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_vpu3.c @@ -0,0 +1,79 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include + +#include "iris_instance.h" +#include "iris_vpu_common.h" +#include "iris_vpu_register_defines.h" + +#define AON_MVP_NOC_RESET 0x0001F000 + +#define WRAPPER_CORE_CLOCK_CONFIG (WRAPPER_BASE_OFFS + 0x88) + +#define CPU_CS_AHB_BRIDGE_SYNC_RESET (CPU_CS_BASE_OFFS + 0x160) + +#define AON_WRAPPER_MVP_NOC_RESET_REQ (AON_MVP_NOC_RESET + 0x000) +#define AON_WRAPPER_MVP_NOC_RESET_ACK (AON_MVP_NOC_RESET + 0x004) + +#define VCODEC_SS_IDLE_STATUSN (VCODEC_BASE_OFFS + 0x70) + +static bool iris_vpu3_hw_power_collapsed(struct iris_core *core) +{ + u32 value = 0, pwr_status = 0; + + value = readl(core->reg_base + WRAPPER_CORE_POWER_STATUS); + pwr_status = value & BIT(1); + + return pwr_status ? false : true; +} + +static void iris_vpu3_power_off_hardware(struct iris_core *core) +{ + u32 reg_val = 0; + u32 value = 0; + int ret, i; + + if (iris_vpu3_hw_power_collapsed(core)) + goto disable_power; + + dev_err(core->dev, "video hw is power on\n"); + + value = readl(core->reg_base + WRAPPER_CORE_CLOCK_CONFIG); + if (value) + writel(0, core->reg_base + WRAPPER_CORE_CLOCK_CONFIG); + + for (i = 0; i < core->iris_platform_data->num_vpp_pipe; i++) { + ret = readl_poll_timeout(core->reg_base + VCODEC_SS_IDLE_STATUSN + 4 * i, + reg_val, reg_val & 0x400000, 2000, 20000); + if (ret) + goto disable_power; + } + + writel(0x3, core->reg_base + AON_WRAPPER_MVP_NOC_RESET_REQ); + + ret = readl_poll_timeout(core->reg_base + AON_WRAPPER_MVP_NOC_RESET_ACK, + reg_val, reg_val & 0x3, 200, 2000); + if (ret) + goto disable_power; + + writel(0x0, core->reg_base + AON_WRAPPER_MVP_NOC_RESET_REQ); + + ret = readl_poll_timeout(core->reg_base + AON_WRAPPER_MVP_NOC_RESET_ACK, + reg_val, !(reg_val & 0x3), 200, 2000); + if (ret) + goto disable_power; + + writel(0x3, core->reg_base + CPU_CS_AHB_BRIDGE_SYNC_RESET); + writel(0x2, core->reg_base + CPU_CS_AHB_BRIDGE_SYNC_RESET); + writel(0x0, core->reg_base + CPU_CS_AHB_BRIDGE_SYNC_RESET); + +disable_power: + iris_vpu_power_off_hw(core); +} + +const struct vpu_ops iris_vpu3_ops = { + .power_off_hw = iris_vpu3_power_off_hardware, +}; diff --git a/drivers/media/platform/qcom/iris/iris_vpu_common.c b/drivers/media/platform/qcom/iris/iris_vpu_common.c index 5930cf105a87..23e4afe24761 100644 --- a/drivers/media/platform/qcom/iris/iris_vpu_common.c +++ b/drivers/media/platform/qcom/iris/iris_vpu_common.c @@ -7,10 +7,11 @@ #include "iris_core.h" #include "iris_vpu_common.h" +#include "iris_vpu_register_defines.h" -#define CPU_BASE_OFFS 0x000A0000 +#define WRAPPER_TZ_BASE_OFFS 0x000C0000 +#define AON_BASE_OFFS 0x000E0000 -#define CPU_CS_BASE_OFFS (CPU_BASE_OFFS) #define CPU_IC_BASE_OFFS (CPU_BASE_OFFS) #define CPU_CS_A2HSOFTINTCLR (CPU_CS_BASE_OFFS + 0x1C) @@ -18,7 +19,9 @@ #define CTRL_INIT (CPU_CS_BASE_OFFS + 0x48) #define CTRL_STATUS (CPU_CS_BASE_OFFS + 0x4C) +#define CTRL_INIT_IDLE_MSG_BMSK 0x40000000 #define CTRL_ERROR_STATUS__M 0xfe +#define CTRL_STATUS_PC_READY 0x100 #define QTBL_INFO (CPU_CS_BASE_OFFS + 0x50) #define QTBL_ADDR (CPU_CS_BASE_OFFS + 0x54) @@ -33,11 +36,35 @@ #define CPU_IC_SOFTINT (CPU_IC_BASE_OFFS + 0x150) #define CPU_IC_SOFTINT_H2A_SHFT 0x0 -#define WRAPPER_BASE_OFFS 0x000B0000 #define WRAPPER_INTR_STATUS (WRAPPER_BASE_OFFS + 0x0C) #define WRAPPER_INTR_STATUS_A2HWD_BMSK 0x8 #define WRAPPER_INTR_STATUS_A2H_BMSK 0x4 -#define CTRL_INIT_IDLE_MSG_BMSK 0x40000000 + +#define WRAPPER_INTR_MASK (WRAPPER_BASE_OFFS + 0x10) +#define WRAPPER_INTR_MASK_A2HWD_BMSK 0x8 +#define WRAPPER_INTR_MASK_A2HCPU_BMSK 0x4 + +#define WRAPPER_DEBUG_BRIDGE_LPI_CONTROL (WRAPPER_BASE_OFFS + 0x54) +#define WRAPPER_DEBUG_BRIDGE_LPI_STATUS (WRAPPER_BASE_OFFS + 0x58) +#define WRAPPER_IRIS_CPU_NOC_LPI_CONTROL (WRAPPER_BASE_OFFS + 0x5C) +#define WRAPPER_IRIS_CPU_NOC_LPI_STATUS (WRAPPER_BASE_OFFS + 0x60) + +#define WRAPPER_TZ_CPU_STATUS (WRAPPER_TZ_BASE_OFFS + 0x10) +#define WRAPPER_TZ_CTL_AXI_CLOCK_CONFIG (WRAPPER_TZ_BASE_OFFS + 0x14) +#define WRAPPER_TZ_QNS4PDXFIFO_RESET (WRAPPER_TZ_BASE_OFFS + 0x18) + +#define AON_WRAPPER_MVP_NOC_LPI_CONTROL (AON_BASE_OFFS) +#define AON_WRAPPER_MVP_NOC_LPI_STATUS (AON_BASE_OFFS + 0x4) + +static void iris_vpu_interrupt_init(struct iris_core *core) +{ + u32 mask_val; + + mask_val = readl(core->reg_base + WRAPPER_INTR_MASK); + mask_val &= ~(WRAPPER_INTR_MASK_A2HWD_BMSK | + WRAPPER_INTR_MASK_A2HCPU_BMSK); + writel(mask_val, core->reg_base + WRAPPER_INTR_MASK); +} static void iris_vpu_setup_ucregion_memory_map(struct iris_core *core) { @@ -127,3 +154,218 @@ int iris_vpu_watchdog(struct iris_core *core, u32 intr_status) return 0; } + +int iris_vpu_prepare_pc(struct iris_core *core) +{ + u32 wfi_status = 0, idle_status = 0, pc_ready = 0; + u32 ctrl_status = 0; + int val = 0; + int ret; + + ctrl_status = readl(core->reg_base + CTRL_STATUS); + pc_ready = ctrl_status & CTRL_STATUS_PC_READY; + idle_status = ctrl_status & BIT(30); + if (pc_ready) + return 0; + + wfi_status = readl(core->reg_base + WRAPPER_TZ_CPU_STATUS); + wfi_status &= BIT(0); + if (!wfi_status || !idle_status) + goto skip_power_off; + + ret = core->hfi_ops->sys_pc_prep(core); + if (ret) { + dev_err(core->dev, "failed to prepare iris for power off\n"); + goto skip_power_off; + } + + ret = readl_poll_timeout(core->reg_base + CTRL_STATUS, val, + val & CTRL_STATUS_PC_READY, 250, 2500); + if (ret) + goto skip_power_off; + + ret = readl_poll_timeout(core->reg_base + WRAPPER_TZ_CPU_STATUS, + val, val & BIT(0), 250, 2500); + if (ret) + goto skip_power_off; + + return 0; + +skip_power_off: + ctrl_status = readl(core->reg_base + CTRL_STATUS); + wfi_status = readl(core->reg_base + WRAPPER_TZ_CPU_STATUS); + wfi_status &= BIT(0); + dev_err(core->dev, "skip power collapse, wfi=%#x, idle=%#x, pcr=%#x, ctrl=%#x)\n", + wfi_status, idle_status, pc_ready, ctrl_status); + + return -EAGAIN; +} + +static int iris_vpu_power_off_controller(struct iris_core *core) +{ + int val = 0; + int ret; + + writel(0x3, core->reg_base + CPU_CS_X2RPMH); + + writel(0x1, core->reg_base + AON_WRAPPER_MVP_NOC_LPI_CONTROL); + + ret = readl_poll_timeout(core->reg_base + AON_WRAPPER_MVP_NOC_LPI_STATUS, + val, val & BIT(0), 200, 2000); + if (ret) + goto disable_power; + + writel(0x1, core->reg_base + WRAPPER_IRIS_CPU_NOC_LPI_CONTROL); + + ret = readl_poll_timeout(core->reg_base + WRAPPER_IRIS_CPU_NOC_LPI_STATUS, + val, val & BIT(0), 200, 2000); + if (ret) + goto disable_power; + + writel(0x0, core->reg_base + WRAPPER_DEBUG_BRIDGE_LPI_CONTROL); + + ret = readl_poll_timeout(core->reg_base + WRAPPER_DEBUG_BRIDGE_LPI_STATUS, + val, val == 0, 200, 2000); + if (ret) + goto disable_power; + + writel(0x3, core->reg_base + WRAPPER_TZ_CTL_AXI_CLOCK_CONFIG); + writel(0x1, core->reg_base + WRAPPER_TZ_QNS4PDXFIFO_RESET); + writel(0x0, core->reg_base + WRAPPER_TZ_QNS4PDXFIFO_RESET); + writel(0x0, core->reg_base + WRAPPER_TZ_CTL_AXI_CLOCK_CONFIG); + +disable_power: + iris_disable_unprepare_clock(core, IRIS_CTRL_CLK); + iris_disable_unprepare_clock(core, IRIS_AXI_CLK); + iris_disable_power_domains(core, core->pmdomain_tbl->pd_devs[IRIS_CTRL_POWER_DOMAIN]); + + return 0; +} + +void iris_vpu_power_off_hw(struct iris_core *core) +{ + dev_pm_genpd_set_hwmode(core->pmdomain_tbl->pd_devs[IRIS_HW_POWER_DOMAIN], false); + iris_disable_power_domains(core, core->pmdomain_tbl->pd_devs[IRIS_HW_POWER_DOMAIN]); + iris_disable_unprepare_clock(core, IRIS_HW_CLK); +} + +void iris_vpu_power_off(struct iris_core *core) +{ + if (!core->power_enabled) + return; + + iris_opp_set_rate(core, 0); + core->iris_platform_data->vpu_ops->power_off_hw(core); + iris_vpu_power_off_controller(core); + iris_unset_icc_bw(core); + + if (!iris_vpu_watchdog(core, core->intr_status)) + disable_irq_nosync(core->irq); + + core->power_enabled = false; +} + +static int iris_vpu_power_on_controller(struct iris_core *core) +{ + int ret; + + ret = iris_enable_power_domains(core, core->pmdomain_tbl->pd_devs[IRIS_CTRL_POWER_DOMAIN]); + if (ret) + return ret; + + ret = iris_reset_ahb2axi_bridge(core); + if (ret) + goto err_disable_power; + + ret = iris_prepare_enable_clock(core, IRIS_AXI_CLK); + if (ret) + goto err_disable_power; + + ret = iris_prepare_enable_clock(core, IRIS_CTRL_CLK); + if (ret) + goto err_disable_clock; + + return 0; + +err_disable_clock: + iris_disable_unprepare_clock(core, IRIS_AXI_CLK); +err_disable_power: + iris_disable_power_domains(core, core->pmdomain_tbl->pd_devs[IRIS_CTRL_POWER_DOMAIN]); + + return ret; +} + +static int iris_vpu_power_on_hw(struct iris_core *core) +{ + int ret; + + ret = iris_enable_power_domains(core, core->pmdomain_tbl->pd_devs[IRIS_HW_POWER_DOMAIN]); + if (ret) + return ret; + + ret = iris_prepare_enable_clock(core, IRIS_HW_CLK); + if (ret) + goto err_disable_power; + + ret = dev_pm_genpd_set_hwmode(core->pmdomain_tbl->pd_devs[IRIS_HW_POWER_DOMAIN], true); + if (ret) + goto err_disable_clock; + + return 0; + +err_disable_clock: + iris_disable_unprepare_clock(core, IRIS_HW_CLK); +err_disable_power: + iris_disable_power_domains(core, core->pmdomain_tbl->pd_devs[IRIS_HW_POWER_DOMAIN]); + + return ret; +} + +int iris_vpu_power_on(struct iris_core *core) +{ + u32 freq = 0; + int ret; + + if (core->power_enabled) + return 0; + + if (core->state != IRIS_CORE_INIT) + return -EINVAL; + + ret = iris_set_icc_bw(core, INT_MAX); + if (ret) + goto err; + + ret = iris_vpu_power_on_controller(core); + if (ret) + goto err_unvote_icc; + + ret = iris_vpu_power_on_hw(core); + if (ret) + goto err_power_off_ctrl; + + core->power_enabled = true; + + freq = core->power.clk_freq ? core->power.clk_freq : + (u32)ULONG_MAX; + + iris_opp_set_rate(core, freq); + + core->iris_platform_data->set_preset_registers(core); + + iris_vpu_interrupt_init(core); + core->intr_status = 0; + enable_irq(core->irq); + + return 0; + +err_power_off_ctrl: + dev_err(core->dev, "power on failed\n"); + iris_vpu_power_off_controller(core); +err_unvote_icc: + iris_unset_icc_bw(core); +err: + core->power_enabled = false; + + return ret; +} diff --git a/drivers/media/platform/qcom/iris/iris_vpu_common.h b/drivers/media/platform/qcom/iris/iris_vpu_common.h index 706b207bc295..525bbb52dd79 100644 --- a/drivers/media/platform/qcom/iris/iris_vpu_common.h +++ b/drivers/media/platform/qcom/iris/iris_vpu_common.h @@ -8,9 +8,20 @@ struct iris_core; +extern const struct vpu_ops iris_vpu2_ops; +extern const struct vpu_ops iris_vpu3_ops; + +struct vpu_ops { + void (*power_off_hw)(struct iris_core *core); +}; + int iris_vpu_boot_firmware(struct iris_core *core); void iris_vpu_raise_interrupt(struct iris_core *core); void iris_vpu_clear_interrupt(struct iris_core *core); int iris_vpu_watchdog(struct iris_core *core, u32 intr_status); +int iris_vpu_prepare_pc(struct iris_core *core); +int iris_vpu_power_on(struct iris_core *core); +void iris_vpu_power_off_hw(struct iris_core *core); +void iris_vpu_power_off(struct iris_core *core); #endif diff --git a/drivers/media/platform/qcom/iris/iris_vpu_register_defines.h b/drivers/media/platform/qcom/iris/iris_vpu_register_defines.h new file mode 100644 index 000000000000..818c81048fe5 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_vpu_register_defines.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_VPU_REGISTER_DEFINES_H_ +#define _IRIS_VPU_REGISTER_DEFINES_H_ + +#define VCODEC_BASE_OFFS 0x00000000 +#define CPU_BASE_OFFS 0x000A0000 +#define WRAPPER_BASE_OFFS 0x000B0000 + +#define CPU_CS_BASE_OFFS (CPU_BASE_OFFS) + +#define WRAPPER_CORE_POWER_STATUS (WRAPPER_BASE_OFFS + 0x80) + +#endif From patchwork Tue Aug 27 10:05:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 822725 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F67519EED0; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; cv=none; b=RP7JgrRfWQE02ufGiotvAQROHcL9kU6Dj95K8yHo4BGXy/p5idCLGqE5b4EbqMYot+ldOw98DDUq3ZqUfznjT/zbZkvKqL0vdcpmGc9lRgGm+yzJTwMN2EdgzNnWkNIKtomP+GdoEOyyKpS8gC4vP1i1cIxhCzOLk4A/iz16+Jc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; c=relaxed/simple; bh=I3tGXkypHF3ecnXstUY12fVCBQjs+Lk5cPvYfleTxXw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=L34LQfm0KSgbSbUCOEpN1K7XtoEz8i+5MAF3K6AycBlC6frk7NDRMCi6YoODmE9tcHA8/qFrwDtsBFnRejSeHNL1GwKHJfOciJYeGNoWbZtudpvoH6va2LG7jsIWivYwOP86YPDTmVTECPWfOx0rY9Fiz4hUsHGF6bmdL6IpSU0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=GIHdIq8U; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="GIHdIq8U" Received: by smtp.kernel.org (Postfix) with ESMTPS id 35F77C4AF1C; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753195; bh=I3tGXkypHF3ecnXstUY12fVCBQjs+Lk5cPvYfleTxXw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=GIHdIq8UD9rw9PKuUTTYXD3dHCplGQXTFDF3DM1JX1Rlo6x0mfMTlOaKLWPIVR2Pd QT3c6bMJXTJyUYRCbBUyAHWq8QhgCS88fQ06Z5D67N+5fE9chulT/7wa2rUWe69j1G jc7C/BnHDNNw35p+wyUbClrp0zZfZjZxs+iWR6YpoeIcNX2SqAtySjvHvDcfoMzhNl 8l2H5m9FGWCDej256UAcKm2N794tmeXwDk5fdZ/w1i7McHw9UBZBcDaTrokf/mC0WP nmKNMHlm/f7ghSz1hBUemBjG52o+N+h7qmPmIwxkqqNoF2DqNSl4SLGQaQNWgZ9rDf zma7hxWpZeL9Q== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 25D9EC5472E; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:36 +0530 Subject: [PATCH v3 11/29] media: iris: implement reqbuf ioctl with vb2_queue_setup Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-11-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=54899; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=uPHnVUh9Ex5xFZOzbNSn7IEqjN2FgJkyoFzRZWdb7CI=; b=+v9gmY6aHgl7F0kBHRzuE9SbySLtxTa4amai1Nw+z57L768U9gugMKzrkHCCHlkF9fYJSCq0k puobzfbD2mGCDTvoC4Jv8FOIMcFeaA1X+MlU36sx3mK4129hl8zbDzP X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal Implement reqbuf IOCTL op and vb2_queue_setup vb2 op in the driver with necessary hooks. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/Makefile | 7 +- drivers/media/platform/qcom/iris/iris_buffer.c | 115 +++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_buffer.h | 107 +++++++++++++++++++ drivers/media/platform/qcom/iris/iris_core.h | 6 ++ drivers/media/platform/qcom/iris/iris_hfi_common.h | 3 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 40 +++++++ .../platform/qcom/iris/iris_hfi_gen1_defines.h | 47 ++++++++- .../platform/qcom/iris/iris_hfi_gen1_response.c | 88 ++++++++++++++-- drivers/media/platform/qcom/iris/iris_hfi_gen2.h | 5 + .../platform/qcom/iris/iris_hfi_gen2_command.c | 106 +++++++++++++++++++ .../platform/qcom/iris/iris_hfi_gen2_defines.h | 34 ++++++ .../platform/qcom/iris/iris_hfi_gen2_packet.c | 39 +++++++ .../platform/qcom/iris/iris_hfi_gen2_packet.h | 7 ++ .../platform/qcom/iris/iris_hfi_gen2_response.c | 109 ++++++++++++++++++- drivers/media/platform/qcom/iris/iris_instance.h | 22 ++++ .../platform/qcom/iris/iris_platform_common.h | 1 + .../platform/qcom/iris/iris_platform_sm8250.c | 1 + .../platform/qcom/iris/iris_platform_sm8550.c | 1 + drivers/media/platform/qcom/iris/iris_probe.c | 2 + drivers/media/platform/qcom/iris/iris_utils.c | 55 ++++++++++ drivers/media/platform/qcom/iris/iris_utils.h | 38 +++++++ drivers/media/platform/qcom/iris/iris_vb2.c | 92 +++++++++++++++++ drivers/media/platform/qcom/iris/iris_vb2.h | 12 +++ drivers/media/platform/qcom/iris/iris_vdec.c | 58 +++++++++++ drivers/media/platform/qcom/iris/iris_vdec.h | 14 +++ drivers/media/platform/qcom/iris/iris_vidc.c | 92 +++++++++++++++++ drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 19 ++++ drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 15 +++ 28 files changed, 1124 insertions(+), 11 deletions(-) diff --git a/drivers/media/platform/qcom/iris/Makefile b/drivers/media/platform/qcom/iris/Makefile index b0447af45891..9c50e29db41e 100644 --- a/drivers/media/platform/qcom/iris/Makefile +++ b/drivers/media/platform/qcom/iris/Makefile @@ -1,4 +1,5 @@ -iris-objs += iris_core.o \ +iris-objs += iris_buffer.o \ + iris_core.o \ iris_firmware.o \ iris_hfi_common.o \ iris_hfi_gen1_command.o \ @@ -13,9 +14,13 @@ iris-objs += iris_core.o \ iris_probe.o \ iris_resources.o \ iris_state.o \ + iris_utils.o \ iris_vidc.o \ + iris_vb2.o \ + iris_vdec.o \ iris_vpu2.o \ iris_vpu3.o \ + iris_vpu_buffer.o \ iris_vpu_common.o \ obj-$(CONFIG_VIDEO_QCOM_IRIS) += iris.o diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c b/drivers/media/platform/qcom/iris/iris_buffer.c new file mode 100644 index 000000000000..a1017ceede7d --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_buffer.c @@ -0,0 +1,115 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include + +#include "iris_buffer.h" +#include "iris_instance.h" + +#define PIXELS_4K 4096 +#define MAX_WIDTH 4096 +#define MAX_HEIGHT 2304 +#define NUM_MBS_4K (DIV_ROUND_UP(MAX_WIDTH, 16) * DIV_ROUND_UP(MAX_HEIGHT, 16)) +#define BASE_RES_MB_MAX 138240 + +/* + * NV12: + * YUV 4:2:0 image with a plane of 8 bit Y samples followed + * by an interleaved U/V plane containing 8 bit 2x2 subsampled + * colour difference samples. + * + * <-------- Y/UV_Stride --------> + * <------- Width -------> + * Y Y Y Y Y Y Y Y Y Y Y Y . . . . ^ ^ + * Y Y Y Y Y Y Y Y Y Y Y Y . . . . | | + * Y Y Y Y Y Y Y Y Y Y Y Y . . . . Height | + * Y Y Y Y Y Y Y Y Y Y Y Y . . . . | y_scanlines + * Y Y Y Y Y Y Y Y Y Y Y Y . . . . | | + * Y Y Y Y Y Y Y Y Y Y Y Y . . . . | | + * Y Y Y Y Y Y Y Y Y Y Y Y . . . . | | + * Y Y Y Y Y Y Y Y Y Y Y Y . . . . V | + * . . . . . . . . . . . . . . . . | + * . . . . . . . . . . . . . . . . | + * . . . . . . . . . . . . . . . . | + * . . . . . . . . . . . . . . . . V + * U V U V U V U V U V U V . . . . ^ + * U V U V U V U V U V U V . . . . | + * U V U V U V U V U V U V . . . . | + * U V U V U V U V U V U V . . . . uv_scanlines + * . . . . . . . . . . . . . . . . | + * . . . . . . . . . . . . . . . . V + * . . . . . . . . . . . . . . . . --> Buffer size alignment + * + * y_stride : Width aligned to 128 + * uv_stride : Width aligned to 128 + * y_scanlines: Height aligned to 32 + * uv_scanlines: Height/2 aligned to 16 + * Total size = align((y_stride * y_scanlines + * + uv_stride * uv_scanlines , 4096) + */ +static u32 iris_output_buffer_size_nv12(struct iris_inst *inst) +{ + u32 y_plane, uv_plane, y_stride, uv_stride, y_scanlines, uv_scanlines; + struct v4l2_format *f; + + f = inst->fmt_dst; + y_stride = ALIGN(f->fmt.pix_mp.width, 128); + uv_stride = ALIGN(f->fmt.pix_mp.width, 128); + y_scanlines = ALIGN(f->fmt.pix_mp.height, 32); + uv_scanlines = ALIGN((f->fmt.pix_mp.height + 1) >> 1, 16); + y_plane = y_stride * y_scanlines; + uv_plane = uv_stride * uv_scanlines; + + return ALIGN(y_plane + uv_plane, PIXELS_4K); +} + +static u32 iris_input_buffer_size(struct iris_inst *inst) +{ + u32 base_res_mbs = NUM_MBS_4K; + u32 frame_size, num_mbs; + u32 div_factor; + + num_mbs = iris_get_mbpf(inst); + if (num_mbs > NUM_MBS_4K) { + div_factor = 4; + base_res_mbs = BASE_RES_MB_MAX; + } else { + base_res_mbs = NUM_MBS_4K; + div_factor = 2; + } + + /* + * frame_size = YUVsize / div_factor + * where YUVsize = resolution_in_MBs * MBs_in_pixel * 3 / 2 + */ + + frame_size = base_res_mbs * (16 * 16) * 3 / 2 / div_factor; + + return ALIGN(frame_size, PIXELS_4K); +} + +int iris_get_buffer_size(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + switch (buffer_type) { + case BUF_INPUT: + return iris_input_buffer_size(inst); + case BUF_OUTPUT: + return iris_output_buffer_size_nv12(inst); + default: + return 0; + } +} + +void iris_vb2_queue_error(struct iris_inst *inst) +{ + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + struct vb2_queue *q; + + q = v4l2_m2m_get_src_vq(m2m_ctx); + vb2_queue_error(q); + q = v4l2_m2m_get_dst_vq(m2m_ctx); + vb2_queue_error(q); +} diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h new file mode 100644 index 000000000000..98844e89e0e3 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_buffer.h @@ -0,0 +1,107 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_BUFFER_H_ +#define _IRIS_BUFFER_H_ + +#include + +struct iris_inst; + +#define to_iris_buffer(ptr) container_of(ptr, struct iris_buffer, vb2) + +/** + * enum iris_buffer_type + * + * BUF_INPUT: input buffer to the iris hardware + * BUF_OUTPUT: output buffer from the iris hardware + * BUF_BIN: buffer to store intermediate bin data + * BUF_ARP: buffer for auto register programming + * BUF_COMV: buffer to store colocated motion vectors + * BUF_NON_COMV: buffer to hold config data for HW + * BUF_LINE: buffer to store decoding/encoding context data for HW + * BUF_DPB: buffer to store display picture buffers for reference + * BUF_PERSIST: buffer to store session context data + * BUF_SCRATCH_1: buffer to store decoding/encoding context data for HW + */ +enum iris_buffer_type { + BUF_INPUT = 1, + BUF_OUTPUT, + BUF_BIN, + BUF_ARP, + BUF_COMV, + BUF_NON_COMV, + BUF_LINE, + BUF_DPB, + BUF_PERSIST, + BUF_SCRATCH_1, + BUF_TYPE_MAX, +}; + +/* + * enum iris_buffer_attributes + * + * BUF_ATTR_DEFERRED: buffer queued by client but not submitted to firmware. + * BUF_ATTR_PENDING_RELEASE: buffers requested to be released from firmware. + * BUF_ATTR_QUEUED: buffers submitted to firmware. + * BUF_ATTR_DEQUEUED: buffers received from firmware. + * BUF_ATTR_BUFFER_DONE: buffers sent back to vb2. + */ +enum iris_buffer_attributes { + BUF_ATTR_DEFERRED = BIT(0), + BUF_ATTR_PENDING_RELEASE = BIT(1), + BUF_ATTR_QUEUED = BIT(2), + BUF_ATTR_DEQUEUED = BIT(3), + BUF_ATTR_BUFFER_DONE = BIT(4), +}; + +/** + * struct iris_buffer + * + * @vb2: v4l2 vb2 buffer + * @list: list head for the iris_buffers structure + * @inst: iris instance structure + * @type: enum for type of iris buffer + * @index: identifier for the iris buffer + * @fd: file descriptor of the buffer + * @buffer_size: accessible buffer size in bytes starting from addr_offset + * @data_offset: accessible buffer offset from base address + * @data_size: data size in bytes + * @device_addr: device address of the buffer + * @kvaddr: kernel virtual address of the buffer + * @dma_attrs: dma attributes + * @flags: buffer flags. It is represented as bit masks. + * @timestamp: timestamp of the buffer in nano seconds (ns) + * @attr: enum for iris buffer attributes + */ +struct iris_buffer { + struct vb2_v4l2_buffer vb2; + struct list_head list; + struct iris_inst *inst; + enum iris_buffer_type type; + u32 index; + int fd; + size_t buffer_size; + u32 data_offset; + size_t data_size; + dma_addr_t device_addr; + void *kvaddr; + unsigned long dma_attrs; + u32 flags; /* V4L2_BUF_FLAG_* */ + u64 timestamp; + enum iris_buffer_attributes attr; +}; + +struct iris_buffers { + struct list_head list; + u32 min_count; + u32 actual_count; + u32 size; +}; + +int iris_get_buffer_size(struct iris_inst *inst, enum iris_buffer_type buffer_type); +void iris_vb2_queue_error(struct iris_inst *inst); + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_core.h b/drivers/media/platform/qcom/iris/iris_core.h index 09a6904c7bb1..1f6eca31928d 100644 --- a/drivers/media/platform/qcom/iris/iris_core.h +++ b/drivers/media/platform/qcom/iris/iris_core.h @@ -28,6 +28,8 @@ * @v4l2_dev: a holder for v4l2 device structure * @vdev_dec: iris video device structure for decoder * @iris_v4l2_file_ops: iris v4l2 file ops + * @iris_v4l2_ioctl_ops: iris v4l2 ioctl ops + * @vb2_ops: iris vb2 ops * @icc_tbl: table of iris interconnects * @icc_count: count of iris interconnects * @pmdomain_tbl: table of iris power domains @@ -55,6 +57,7 @@ * @core_init_done: structure of signal completion for system response * @intr_status: interrupt status * @sys_error_handler: a delayed work for handling system fatal error + * @instances: a list_head of all instances */ struct iris_core { @@ -64,6 +67,8 @@ struct iris_core { struct v4l2_device v4l2_dev; struct video_device *vdev_dec; const struct v4l2_file_operations *iris_v4l2_file_ops; + const struct v4l2_ioctl_ops *iris_v4l2_ioctl_ops; + const struct vb2_ops *iris_vb2_ops; struct icc_bulk_data *icc_tbl; u32 icc_count; struct dev_pm_domain_list *pmdomain_tbl; @@ -91,6 +96,7 @@ struct iris_core { struct completion core_init_done; u32 intr_status; struct delayed_work sys_error_handler; + struct list_head instances; }; int iris_core_init(struct iris_core *core); diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.h b/drivers/media/platform/qcom/iris/iris_hfi_common.h index e050b1ae90fe..f59ce97d5b7e 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_common.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_common.h @@ -9,6 +9,7 @@ #include #include +struct iris_inst; struct iris_core; enum hfi_packet_port_type { @@ -47,6 +48,8 @@ struct iris_hfi_command_ops { int (*sys_image_version)(struct iris_core *core); int (*sys_interframe_powercollapse)(struct iris_core *core); int (*sys_pc_prep)(struct iris_core *core); + int (*session_open)(struct iris_inst *inst); + int (*session_close)(struct iris_inst *inst); }; struct iris_hfi_response_ops { diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c index e778ae33b953..f8bd185bb68b 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c @@ -66,11 +66,51 @@ static int iris_hfi_gen1_sys_pc_prep(struct iris_core *core) return iris_hfi_queue_cmd_write_locked(core, &pkt, pkt.hdr.size); } +static int iris_hfi_gen1_session_open(struct iris_inst *inst) +{ + struct hfi_session_open_pkt packet; + int ret; + + packet.shdr.hdr.size = sizeof(struct hfi_session_open_pkt); + packet.shdr.hdr.pkt_type = HFI_CMD_SYS_SESSION_INIT; + packet.shdr.session_id = inst->session_id; + packet.session_domain = HFI_SESSION_TYPE_DEC; + packet.session_codec = HFI_VIDEO_CODEC_H264; + + reinit_completion(&inst->completion); + + ret = iris_hfi_queue_cmd_write(inst->core, &packet, packet.shdr.hdr.size); + if (ret) + return ret; + + return iris_wait_for_session_response(inst); +} + +static void iris_hfi_gen1_packet_session_cmd(struct iris_inst *inst, + struct hfi_session_pkt *packet, + u32 ptype) +{ + packet->shdr.hdr.size = sizeof(*packet); + packet->shdr.hdr.pkt_type = ptype; + packet->shdr.session_id = inst->session_id; +} + +static int iris_hfi_gen1_session_close(struct iris_inst *inst) +{ + struct hfi_session_pkt packet; + + iris_hfi_gen1_packet_session_cmd(inst, &packet, HFI_CMD_SYS_SESSION_END); + + return iris_hfi_queue_cmd_write(inst->core, &packet, packet.shdr.hdr.size); +} + static const struct iris_hfi_command_ops iris_hfi_gen1_command_ops = { .sys_init = iris_hfi_gen1_sys_init, .sys_image_version = iris_hfi_gen1_sys_image_version, .sys_interframe_powercollapse = iris_hfi_gen1_sys_interframe_powercollapse, .sys_pc_prep = iris_hfi_gen1_sys_pc_prep, + .session_open = iris_hfi_gen1_session_open, + .session_close = iris_hfi_gen1_session_close, }; void iris_hfi_gen1_command_ops_init(struct iris_core *core) diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h index 991d5a5dc792..da52e497b74a 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h @@ -9,19 +9,34 @@ #include #define HFI_VIDEO_ARCH_OX 0x1 + +#define HFI_SESSION_TYPE_DEC 2 + +#define HFI_VIDEO_CODEC_H264 0x00000002 + #define HFI_ERR_NONE 0x0 #define HFI_CMD_SYS_INIT 0x10001 #define HFI_CMD_SYS_PC_PREP 0x10002 #define HFI_CMD_SYS_SET_PROPERTY 0x10005 #define HFI_CMD_SYS_GET_PROPERTY 0x10006 +#define HFI_CMD_SYS_SESSION_INIT 0x10007 +#define HFI_CMD_SYS_SESSION_END 0x10008 -#define HFI_PROPERTY_SYS_CODEC_POWER_PLANE_CTRL 0x5 -#define HFI_PROPERTY_SYS_IMAGE_VERSION 0x6 +#define HFI_ERR_SESSION_UNSUPPORTED_SETTING 0x1008 +#define HFI_ERR_SESSION_UNSUPPORT_BUFFERTYPE 0x1010 +#define HFI_ERR_SESSION_INVALID_SCALE_FACTOR 0x1012 +#define HFI_ERR_SESSION_UPSCALE_NOT_SUPPORTED 0x1013 #define HFI_EVENT_SYS_ERROR 0x1 +#define HFI_EVENT_SESSION_ERROR 0x2 + +#define HFI_PROPERTY_SYS_CODEC_POWER_PLANE_CTRL 0x5 +#define HFI_PROPERTY_SYS_IMAGE_VERSION 0x6 #define HFI_MSG_SYS_INIT 0x20001 +#define HFI_MSG_SYS_SESSION_INIT 0x20006 +#define HFI_MSG_SYS_SESSION_END 0x20007 #define HFI_MSG_SYS_COV 0x20009 #define HFI_MSG_SYS_PROPERTY_INFO 0x2000a @@ -32,6 +47,21 @@ struct hfi_pkt_hdr { u32 pkt_type; }; +struct hfi_session_hdr_pkt { + struct hfi_pkt_hdr hdr; + u32 session_id; +}; + +struct hfi_session_open_pkt { + struct hfi_session_hdr_pkt shdr; + u32 session_domain; + u32 session_codec; +}; + +struct hfi_session_pkt { + struct hfi_session_hdr_pkt shdr; +}; + struct hfi_sys_init_pkt { struct hfi_pkt_hdr hdr; u32 arch_type; @@ -54,7 +84,7 @@ struct hfi_sys_pc_prep_pkt { }; struct hfi_msg_event_notify_pkt { - struct hfi_pkt_hdr hdr; + struct hfi_session_hdr_pkt shdr; u32 event_id; u32 event_data1; u32 event_data2; @@ -68,6 +98,17 @@ struct hfi_msg_sys_init_done_pkt { u32 data[]; }; +struct hfi_msg_session_hdr_pkt { + struct hfi_session_hdr_pkt shdr; + u32 error_type; +}; + +struct hfi_msg_session_init_done_pkt { + struct hfi_msg_session_hdr_pkt shdr; + u32 num_properties; + u32 data[]; +}; + struct hfi_msg_sys_property_info_pkt { struct hfi_pkt_hdr hdr; u32 num_properties; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c index 3eb2ce99c614..c9b87ff4db3a 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c @@ -13,13 +13,54 @@ iris_hfi_gen1_sys_event_notify(struct iris_core *core, void *packet) struct hfi_msg_event_notify_pkt *pkt = packet; if (pkt->event_id == HFI_EVENT_SYS_ERROR) - dev_err(core->dev, "sys error (type: %x, data1:%x, data2:%x)\n", - pkt->event_id, pkt->event_data1, pkt->event_data2); + dev_err(core->dev, "sys error (type: %x, session id:%x, data1:%x, data2:%x)\n", + pkt->event_id, pkt->shdr.session_id, pkt->event_data1, + pkt->event_data2); iris_change_core_state(core, IRIS_CORE_ERROR); schedule_delayed_work(&core->sys_error_handler, msecs_to_jiffies(10)); } +static void +iris_hfi_gen1_event_session_error(struct iris_inst *inst, struct hfi_msg_event_notify_pkt *pkt) +{ + switch (pkt->event_data1) { + /* non fatal session errors */ + case HFI_ERR_SESSION_INVALID_SCALE_FACTOR: + case HFI_ERR_SESSION_UNSUPPORT_BUFFERTYPE: + case HFI_ERR_SESSION_UNSUPPORTED_SETTING: + case HFI_ERR_SESSION_UPSCALE_NOT_SUPPORTED: + dev_dbg(inst->core->dev, "session error: event id:%x, session id:%x\n", + pkt->event_data1, pkt->shdr.session_id); + break; + /* fatal session errors */ + default: + /* + * firmware fills event_data2 as an additional information about the + * hfi command for which session error has ouccured. + */ + dev_err(inst->core->dev, + "session error for command: %x, event id:%x, session id:%x\n", + pkt->event_data2, pkt->event_data1, + pkt->shdr.session_id); + iris_vb2_queue_error(inst); + break; + } +} + +static void iris_hfi_gen1_session_event_notify(struct iris_inst *inst, void *packet) +{ + struct hfi_msg_event_notify_pkt *pkt = packet; + + switch (pkt->event_id) { + case HFI_EVENT_SESSION_ERROR: + iris_hfi_gen1_event_session_error(inst, pkt); + break; + default: + break; + } +} + static void iris_hfi_gen1_sys_init_done(struct iris_core *core, void *packet) { struct hfi_msg_sys_init_done_pkt *pkt = packet; @@ -99,6 +140,14 @@ static const struct iris_hfi_gen1_response_pkt_info pkt_infos[] = { .pkt = HFI_MSG_SYS_PROPERTY_INFO, .pkt_sz = sizeof(struct hfi_msg_sys_property_info_pkt), }, + { + .pkt = HFI_MSG_SYS_SESSION_INIT, + .pkt_sz = sizeof(struct hfi_msg_session_init_done_pkt), + }, + { + .pkt = HFI_MSG_SYS_SESSION_END, + .pkt_sz = sizeof(struct hfi_msg_session_hdr_pkt), + }, }; static void iris_hfi_gen1_handle_response(struct iris_core *core, void *response) @@ -106,6 +155,7 @@ static void iris_hfi_gen1_handle_response(struct iris_core *core, void *response const struct iris_hfi_gen1_response_pkt_info *pkt_info; struct device *dev = core->dev; struct hfi_pkt_hdr *hdr; + struct iris_inst *inst; bool found = false; unsigned int i; @@ -126,12 +176,38 @@ static void iris_hfi_gen1_handle_response(struct iris_core *core, void *response return; } - if (hdr->pkt_type == HFI_MSG_SYS_INIT) + if (hdr->pkt_type == HFI_MSG_SYS_INIT) { iris_hfi_gen1_sys_init_done(core, hdr); - else if (hdr->pkt_type == HFI_MSG_SYS_PROPERTY_INFO) + } else if (hdr->pkt_type == HFI_MSG_SYS_PROPERTY_INFO) { iris_hfi_gen1_sys_property_info(core, hdr); - else if (hdr->pkt_type == HFI_MSG_EVENT_NOTIFY) - iris_hfi_gen1_sys_event_notify(core, hdr); + } else if (hdr->pkt_type == HFI_MSG_EVENT_NOTIFY) { + struct hfi_session_pkt *pkt; + + pkt = (struct hfi_session_pkt *)hdr; + inst = iris_get_instance(core, pkt->shdr.session_id); + if (inst) { + mutex_lock(&inst->lock); + iris_hfi_gen1_session_event_notify(inst, hdr); + mutex_unlock(&inst->lock); + } else { + iris_hfi_gen1_sys_event_notify(core, hdr); + } + } else { + struct hfi_session_pkt *pkt; + + pkt = (struct hfi_session_pkt *)hdr; + inst = iris_get_instance(core, pkt->shdr.session_id); + if (!inst) { + dev_warn(dev, "no valid instance(pkt session_id:%x, pkt:%x)\n", + pkt->shdr.session_id, + pkt_info ? pkt_info->pkt : 0); + return; + } + + mutex_lock(&inst->lock); + complete(&inst->completion); + mutex_unlock(&inst->lock); + } } static void iris_hfi_gen1_flush_debug_queue(struct iris_core *core, u8 *packet) diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2.h index 6ec83984fda9..76f0c9032a92 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2.h @@ -10,13 +10,18 @@ struct iris_core; +#define to_iris_inst_hfi_gen2(ptr) \ + container_of(ptr, struct iris_inst_hfi_gen2, inst) + /** * struct iris_inst_hfi_gen2 - holds per video instance parameters for hfi_gen2 * * @inst: pointer to iris_instance structure + * @packet: HFI packet */ struct iris_inst_hfi_gen2 { struct iris_inst inst; + struct iris_hfi_header *packet; }; void iris_hfi_gen2_command_ops_init(struct iris_core *core); diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c index 0871c0bdea90..a74114b0761a 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c @@ -85,11 +85,117 @@ static int iris_hfi_gen2_sys_pc_prep(struct iris_core *core) return ret; } +static int iris_hfi_gen2_session_set_codec(struct iris_inst *inst) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + u32 codec; + + codec = HFI_CODEC_DECODE_AVC; + iris_hfi_gen2_packet_session_property(inst, + HFI_PROP_CODEC, + HFI_HOST_FLAGS_NONE, + HFI_PORT_NONE, + HFI_PAYLOAD_U32_ENUM, + &codec, + sizeof(u32)); + + return iris_hfi_queue_cmd_write(inst->core, inst_hfi_gen2->packet, + inst_hfi_gen2->packet->size); +} + +static int iris_hfi_gen2_session_set_default_header(struct iris_inst *inst) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + u32 default_header = false; + + iris_hfi_gen2_packet_session_property(inst, + HFI_PROP_DEC_DEFAULT_HEADER, + HFI_HOST_FLAGS_NONE, + HFI_PORT_BITSTREAM, + HFI_PAYLOAD_U32, + &default_header, + sizeof(u32)); + + return iris_hfi_queue_cmd_write(inst->core, inst_hfi_gen2->packet, + inst_hfi_gen2->packet->size); +} + +static int iris_hfi_gen2_session_open(struct iris_inst *inst) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + int ret; + + inst_hfi_gen2->packet = kzalloc(4096, GFP_KERNEL); + if (!inst_hfi_gen2->packet) + return -ENOMEM; + + iris_hfi_gen2_packet_session_command(inst, + HFI_CMD_OPEN, + HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED, + HFI_PORT_NONE, + 0, + HFI_PAYLOAD_U32, + &inst->session_id, + sizeof(u32)); + + ret = iris_hfi_queue_cmd_write(inst->core, inst_hfi_gen2->packet, + inst_hfi_gen2->packet->size); + if (ret) + goto fail_free_packet; + + ret = iris_hfi_gen2_session_set_codec(inst); + if (ret) + goto fail_free_packet; + + ret = iris_hfi_gen2_session_set_default_header(inst); + if (ret) + goto fail_free_packet; + + return ret; + +fail_free_packet: + kfree(inst_hfi_gen2->packet); + inst_hfi_gen2->packet = NULL; + + return ret; +} + +static int iris_hfi_gen2_session_close(struct iris_inst *inst) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + int ret; + + if (!inst_hfi_gen2->packet) + return -EINVAL; + + iris_hfi_gen2_packet_session_command(inst, + HFI_CMD_CLOSE, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED | + HFI_HOST_FLAGS_NON_DISCARDABLE), + HFI_PORT_NONE, + inst->session_id, + HFI_PAYLOAD_NONE, + NULL, + 0); + + ret = iris_hfi_queue_cmd_write(inst->core, inst_hfi_gen2->packet, + inst_hfi_gen2->packet->size); + + kfree(inst_hfi_gen2->packet); + inst_hfi_gen2->packet = NULL; + + return ret; +} + static const struct iris_hfi_command_ops iris_hfi_gen2_command_ops = { .sys_init = iris_hfi_gen2_sys_init, .sys_image_version = iris_hfi_gen2_sys_image_version, .sys_interframe_powercollapse = iris_hfi_gen2_sys_interframe_powercollapse, .sys_pc_prep = iris_hfi_gen2_sys_pc_prep, + .session_open = iris_hfi_gen2_session_open, + .session_close = iris_hfi_gen2_session_close, }; void iris_hfi_gen2_command_ops_init(struct iris_core *core) diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h index 4104479c7251..18cc9365ab75 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h @@ -13,6 +13,8 @@ #define HFI_CMD_BEGIN 0x01000000 #define HFI_CMD_INIT 0x01000001 #define HFI_CMD_POWER_COLLAPSE 0x01000002 +#define HFI_CMD_OPEN 0x01000003 +#define HFI_CMD_CLOSE 0x01000004 #define HFI_CMD_END 0x01FFFFFF #define HFI_PROP_BEGIN 0x03000000 @@ -25,12 +27,44 @@ #define HFI_PROP_UBWC_BANK_SWZL_LEVEL2 0x03000007 #define HFI_PROP_UBWC_BANK_SWZL_LEVEL3 0x03000008 #define HFI_PROP_UBWC_BANK_SPREADING 0x03000009 +#define HFI_PROP_CODEC 0x03000100 +#define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168 #define HFI_PROP_END 0x03FFFFFF +#define HFI_SESSION_ERROR_BEGIN 0x04000000 +#define HFI_ERROR_UNKNOWN_SESSION 0x04000001 +#define HFI_ERROR_MAX_SESSIONS 0x04000002 +#define HFI_ERROR_FATAL 0x04000003 +#define HFI_ERROR_INVALID_STATE 0x04000004 +#define HFI_ERROR_INSUFFICIENT_RESOURCES 0x04000005 +#define HFI_ERROR_BUFFER_NOT_SET 0x04000006 +#define HFI_SESSION_ERROR_END 0x04FFFFFF + #define HFI_SYSTEM_ERROR_BEGIN 0x05000000 #define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001 #define HFI_SYSTEM_ERROR_END 0x05FFFFFF +enum hfi_codec_type { + HFI_CODEC_DECODE_AVC = 1, + HFI_CODEC_ENCODE_AVC = 2, +}; + +enum hfi_buffer_type { + HFI_BUFFER_BITSTREAM = 0x00000001, + HFI_BUFFER_RAW = 0x00000002, + HFI_BUFFER_METADATA = 0x00000003, + HFI_BUFFER_SUBCACHE = 0x00000004, + HFI_BUFFER_PARTIAL_DATA = 0x00000005, + HFI_BUFFER_DPB = 0x00000006, + HFI_BUFFER_BIN = 0x00000007, + HFI_BUFFER_LINE = 0x00000008, + HFI_BUFFER_ARP = 0x00000009, + HFI_BUFFER_COMV = 0x0000000A, + HFI_BUFFER_NON_COMV = 0x0000000B, + HFI_BUFFER_PERSIST = 0x0000000C, + HFI_BUFFER_VPSS = 0x0000000D, +}; + enum hfi_packet_firmware_flags { HFI_FW_FLAGS_SUCCESS = 0x00000001, HFI_FW_FLAGS_INFORMATION = 0x00000002, diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c index 9ea26328a300..09e7c07fdc5f 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c @@ -146,6 +146,45 @@ void iris_hfi_gen2_packet_image_version(struct iris_core *core, struct iris_hfi_ NULL, 0); } +void iris_hfi_gen2_packet_session_command(struct iris_inst *inst, u32 pkt_type, + u32 flags, u32 port, u32 session_id, + u32 payload_type, void *payload, + u32 payload_size) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + struct iris_core *core = inst->core; + + iris_hfi_gen2_create_header(inst_hfi_gen2->packet, session_id, core->header_id++); + + iris_hfi_gen2_create_packet(inst_hfi_gen2->packet, + pkt_type, + flags, + payload_type, + port, + core->packet_id++, + payload, + payload_size); +} + +void iris_hfi_gen2_packet_session_property(struct iris_inst *inst, + u32 pkt_type, u32 flags, u32 port, + u32 payload_type, void *payload, u32 payload_size) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + struct iris_core *core = inst->core; + + iris_hfi_gen2_create_header(inst_hfi_gen2->packet, inst->session_id, core->header_id++); + + iris_hfi_gen2_create_packet(inst_hfi_gen2->packet, + pkt_type, + flags, + payload_type, + port, + core->packet_id++, + payload, + payload_size); +} + void iris_hfi_gen2_packet_sys_interframe_powercollapse(struct iris_core *core, struct iris_hfi_header *hdr) { diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h index 163295783b7d..120592322e78 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h @@ -63,6 +63,13 @@ struct iris_hfi_packet { void iris_hfi_gen2_packet_sys_init(struct iris_core *core, struct iris_hfi_header *hdr); void iris_hfi_gen2_packet_image_version(struct iris_core *core, struct iris_hfi_header *hdr); +void iris_hfi_gen2_packet_session_command(struct iris_inst *inst, u32 pkt_type, + u32 flags, u32 port, u32 session_id, + u32 payload_type, void *payload, + u32 payload_size); +void iris_hfi_gen2_packet_session_property(struct iris_inst *inst, + u32 pkt_type, u32 flags, u32 port, + u32 payload_type, void *payload, u32 payload_size); void iris_hfi_gen2_packet_sys_interframe_powercollapse(struct iris_core *core, struct iris_hfi_header *hdr); void iris_hfi_gen2_packet_sys_pc_prep(struct iris_core *core, struct iris_hfi_header *hdr); diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c index e208a5ae664a..dce2cf04a856 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c @@ -14,6 +14,17 @@ struct iris_hfi_gen2_core_hfi_range { int (*handle)(struct iris_core *core, struct iris_hfi_packet *pkt); }; +struct iris_hfi_gen2_inst_hfi_range { + u32 begin; + u32 end; + int (*handle)(struct iris_inst *inst, struct iris_hfi_packet *pkt); +}; + +struct iris_hfi_gen2_packet_handle { + enum hfi_buffer_type type; + int (*handle)(struct iris_inst *inst, struct iris_hfi_packet *pkt); +}; + static int iris_hfi_gen2_validate_packet(u8 *response_pkt, u8 *core_resp_pkt) { u32 response_pkt_size = 0; @@ -57,6 +68,43 @@ static int iris_hfi_gen2_validate_hdr_packet(struct iris_core *core, struct iris return ret; } +static int iris_hfi_gen2_handle_session_error(struct iris_inst *inst, + struct iris_hfi_packet *pkt) +{ + struct iris_core *core = inst->core; + char *error; + + switch (pkt->type) { + case HFI_ERROR_MAX_SESSIONS: + error = "exceeded max sessions"; + break; + case HFI_ERROR_UNKNOWN_SESSION: + error = "unknown session id"; + break; + case HFI_ERROR_INVALID_STATE: + error = "invalid operation for current state"; + break; + case HFI_ERROR_INSUFFICIENT_RESOURCES: + error = "insufficient resources"; + break; + case HFI_ERROR_BUFFER_NOT_SET: + error = "internal buffers not set"; + break; + case HFI_ERROR_FATAL: + error = "fatal error"; + break; + default: + error = "unknown"; + break; + } + + dev_err(core->dev, "session error received %#x: %s\n", + pkt->type, error); + iris_vb2_queue_error(inst); + + return 0; +} + static int iris_hfi_gen2_handle_system_error(struct iris_core *core, struct iris_hfi_packet *pkt) { @@ -81,6 +129,22 @@ static int iris_hfi_gen2_handle_system_init(struct iris_core *core, return 0; } +static int iris_hfi_gen2_handle_session_command(struct iris_inst *inst, + struct iris_hfi_packet *pkt) +{ + int ret = 0; + + switch (pkt->type) { + case HFI_CMD_CLOSE: + complete(&inst->completion); + break; + default: + break; + } + + return ret; +} + static int iris_hfi_gen2_handle_image_version_property(struct iris_core *core, struct iris_hfi_packet *pkt) { @@ -163,6 +227,46 @@ static int iris_hfi_gen2_handle_system_response(struct iris_core *core, return ret; } +static int iris_hfi_gen2_handle_session_response(struct iris_core *core, + struct iris_hfi_header *hdr) +{ + struct iris_hfi_packet *packet; + struct iris_inst *inst; + u8 *pkt, *start_pkt; + int ret = 0; + int i, j; + static const struct iris_hfi_gen2_inst_hfi_range range[] = { + {HFI_SESSION_ERROR_BEGIN, HFI_SESSION_ERROR_END, + iris_hfi_gen2_handle_session_error}, + {HFI_CMD_BEGIN, HFI_CMD_END, + iris_hfi_gen2_handle_session_command }, + }; + + inst = iris_get_instance(core, hdr->session_id); + if (!inst) + return -EINVAL; + + mutex_lock(&inst->lock); + + start_pkt = (u8 *)((u8 *)hdr + sizeof(*hdr)); + for (i = 0; i < ARRAY_SIZE(range); i++) { + pkt = start_pkt; + for (j = 0; j < hdr->num_packets; j++) { + packet = (struct iris_hfi_packet *)pkt; + if (packet->flags & HFI_FW_FLAGS_SESSION_ERROR) + iris_hfi_gen2_handle_session_error(inst, packet); + + if (packet->type > range[i].begin && packet->type < range[i].end) + ret = range[i].handle(inst, packet); + pkt += packet->size; + } + } + + mutex_unlock(&inst->lock); + + return ret; +} + static int iris_hfi_gen2_handle_response(struct iris_core *core, void *response) { struct iris_hfi_header *hdr; @@ -173,7 +277,10 @@ static int iris_hfi_gen2_handle_response(struct iris_core *core, void *response) if (ret) return iris_hfi_gen2_handle_system_error(core, NULL); - return iris_hfi_gen2_handle_system_response(core, hdr); + if (!hdr->session_id) + return iris_hfi_gen2_handle_system_response(core, hdr); + else + return iris_hfi_gen2_handle_session_response(core, hdr); } static void iris_hfi_gen2_flush_debug_queue(struct iris_core *core, u8 *packet) diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h index 63cb9d70166f..bb43119af352 100644 --- a/drivers/media/platform/qcom/iris/iris_instance.h +++ b/drivers/media/platform/qcom/iris/iris_instance.h @@ -6,24 +6,46 @@ #ifndef _IRIS_INSTANCE_H_ #define _IRIS_INSTANCE_H_ +#include + +#include "iris_buffer.h" #include "iris_core.h" +#include "iris_utils.h" /** * struct iris_inst - holds per video instance parameters * + * @list: used for attach an instance to the core * @core: pointer to core structure + * @session_id: id of current video session * @ctx_q_lock: lock to serialize queues related ioctls * @lock: lock to seralise forward and reverse threads * @fh: reference of v4l2 file handler + * @fmt_src: structure of v4l2_format for source + * @fmt_dst: structure of v4l2_format for destination + * @crop: structure of crop info + * @completions: structure of signal completions + * @buffers: array of different iris buffers + * @fw_min_count: minimnum count of buffers needed by fw + * @once_per_session_set: boolean to set once per session property * @m2m_dev: a reference to m2m device structure * @m2m_ctx: a reference to m2m context structure */ struct iris_inst { + struct list_head list; struct iris_core *core; + u32 session_id; struct mutex ctx_q_lock;/* lock to serialize queues related ioctls */ struct mutex lock; /* lock to serialize forward and reverse threads */ struct v4l2_fh fh; + struct v4l2_format *fmt_src; + struct v4l2_format *fmt_dst; + struct iris_hfi_rect_desc crop; + struct completion completion; + struct iris_buffers buffers[BUF_TYPE_MAX]; + u32 fw_min_count; + bool once_per_session_set; struct v4l2_m2m_dev *m2m_dev; struct v4l2_m2m_ctx *m2m_ctx; }; diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h index 899a696a931d..754cccc638a5 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_common.h +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h @@ -75,6 +75,7 @@ struct iris_platform_data { u32 hw_response_timeout; struct ubwc_config_data *ubwc_config; u32 num_vpp_pipe; + u32 max_session_count; }; #endif diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c index 1adbafa373a5..cbc5e84641f6 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c @@ -60,4 +60,5 @@ struct iris_platform_data sm8250_data = { .tz_cp_config_data = &tz_cp_config_sm8250, .hw_response_timeout = HW_RESPONSE_TIMEOUT_VALUE, .num_vpp_pipe = 4, + .max_session_count = 16, }; diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c index 950ccdccde31..57fe9986b8cf 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c @@ -74,4 +74,5 @@ struct iris_platform_data sm8550_data = { .hw_response_timeout = HW_RESPONSE_TIMEOUT_VALUE, .ubwc_config = &ubwc_config_sm8550, .num_vpp_pipe = 4, + .max_session_count = 16, }; diff --git a/drivers/media/platform/qcom/iris/iris_probe.c b/drivers/media/platform/qcom/iris/iris_probe.c index 3222e9116551..5d492b3919cc 100644 --- a/drivers/media/platform/qcom/iris/iris_probe.c +++ b/drivers/media/platform/qcom/iris/iris_probe.c @@ -36,6 +36,7 @@ static int iris_register_video_device(struct iris_core *core) strscpy(vdev->name, "qcom-iris-decoder", sizeof(vdev->name)); vdev->release = video_device_release; vdev->fops = core->iris_v4l2_file_ops; + vdev->ioctl_ops = core->iris_v4l2_ioctl_ops; vdev->vfl_dir = VFL_DIR_M2M; vdev->v4l2_dev = &core->v4l2_dev; vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING; @@ -102,6 +103,7 @@ static int iris_probe(struct platform_device *pdev) if (!core->response_packet) return -ENOMEM; + INIT_LIST_HEAD(&core->instances); INIT_DELAYED_WORK(&core->sys_error_handler, iris_sys_error_handler); core->reg_base = devm_platform_ioremap_resource(pdev, 0); diff --git a/drivers/media/platform/qcom/iris/iris_utils.c b/drivers/media/platform/qcom/iris/iris_utils.c new file mode 100644 index 000000000000..d7e7c9852aca --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_utils.c @@ -0,0 +1,55 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include + +#include "iris_instance.h" +#include "iris_utils.h" + +int iris_get_mbpf(struct iris_inst *inst) +{ + struct v4l2_format *inp_f; + int height, width; + + inp_f = inst->fmt_src; + width = max(inp_f->fmt.pix_mp.width, inst->crop.width); + height = max(inp_f->fmt.pix_mp.height, inst->crop.height); + + return NUM_MBS_PER_FRAME(height, width); +} + +int iris_wait_for_session_response(struct iris_inst *inst) +{ + struct iris_core *core = inst->core; + u32 hw_response_timeout_val; + int ret; + + hw_response_timeout_val = core->iris_platform_data->hw_response_timeout; + + mutex_unlock(&inst->lock); + ret = wait_for_completion_timeout(&inst->completion, + msecs_to_jiffies(hw_response_timeout_val)); + mutex_lock(&inst->lock); + if (!ret) + return -ETIMEDOUT; + + return 0; +} + +struct iris_inst *iris_get_instance(struct iris_core *core, u32 session_id) +{ + struct iris_inst *inst = NULL; + + mutex_lock(&core->lock); + list_for_each_entry(inst, &core->instances, list) { + if (inst->session_id == session_id) { + mutex_unlock(&core->lock); + return inst; + } + } + mutex_unlock(&core->lock); + + return NULL; +} diff --git a/drivers/media/platform/qcom/iris/iris_utils.h b/drivers/media/platform/qcom/iris/iris_utils.h new file mode 100644 index 000000000000..1c1e109d9b5b --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_utils.h @@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_UTILS_H_ +#define _IRIS_UTILS_H_ + +struct iris_core; +#include "iris_buffer.h" + +struct iris_hfi_rect_desc { + u32 left; + u32 top; + u32 width; + u32 height; +}; + +#define NUM_MBS_PER_FRAME(height, width) \ + (DIV_ROUND_UP(height, 16) * DIV_ROUND_UP(width, 16)) + +static inline enum iris_buffer_type iris_v4l2_type_to_driver(u32 type) +{ + switch (type) { + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: + return BUF_INPUT; + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: + return BUF_OUTPUT; + default: + return -EINVAL; + } +} + +int iris_get_mbpf(struct iris_inst *inst); +struct iris_inst *iris_get_instance(struct iris_core *core, u32 session_id); +int iris_wait_for_session_response(struct iris_inst *inst); + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_vb2.c b/drivers/media/platform/qcom/iris/iris_vb2.c new file mode 100644 index 000000000000..3fd18b6773fd --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_vb2.c @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_buffer.h" +#include "iris_instance.h" +#include "iris_vb2.h" +#include "iris_vpu_buffer.h" + +int iris_vb2_queue_setup(struct vb2_queue *q, + unsigned int *num_buffers, unsigned int *num_planes, + unsigned int sizes[], struct device *alloc_devs[]) +{ + enum iris_buffer_type buffer_type = 0; + struct iris_buffers *buffers; + struct iris_inst *inst; + struct iris_core *core; + struct v4l2_format *f; + int ret = 0; + + if (!q || !num_buffers || !num_planes || !sizes) + return -EINVAL; + + inst = vb2_get_drv_priv(q); + if (!inst || !inst->core) + return -EINVAL; + + mutex_lock(&inst->lock); + + core = inst->core; + f = V4L2_TYPE_IS_OUTPUT(q->type) ? inst->fmt_src : inst->fmt_dst; + + if (*num_planes) { + if (*num_planes != f->fmt.pix_mp.num_planes || + sizes[0] < f->fmt.pix_mp.plane_fmt[0].sizeimage) { + ret = -EINVAL; + goto unlock; + } + } + + buffer_type = iris_v4l2_type_to_driver(q->type); + if (buffer_type == -EINVAL) { + ret = -EINVAL; + goto unlock; + } + + if (!inst->once_per_session_set) { + inst->once_per_session_set = true; + + mutex_lock(&core->lock); + if (core->state == IRIS_CORE_ERROR) { + mutex_unlock(&core->lock); + ret = -EIO; + goto unlock; + } + mutex_unlock(&core->lock); + + ret = core->hfi_ops->session_open(inst); + if (ret) { + ret = -EINVAL; + dev_err(core->dev, "session open failed\n"); + goto unlock; + } + } + + buffers = &inst->buffers[buffer_type]; + if (!buffers) { + ret = -EINVAL; + goto unlock; + } + + buffers->min_count = iris_vpu_buf_count(inst, buffer_type); + if (*num_buffers < buffers->min_count) + *num_buffers = buffers->min_count; + buffers->actual_count = *num_buffers; + *num_planes = 1; + + buffers->size = iris_get_buffer_size(inst, buffer_type); + + if (sizes[0] < buffers->size) { + f->fmt.pix_mp.plane_fmt[0].sizeimage = buffers->size; + sizes[0] = f->fmt.pix_mp.plane_fmt[0].sizeimage; + } + + q->dev = core->dev; + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} diff --git a/drivers/media/platform/qcom/iris/iris_vb2.h b/drivers/media/platform/qcom/iris/iris_vb2.h new file mode 100644 index 000000000000..78157a97b86e --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_vb2.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_VB2_H_ +#define _IRIS_VB2_H_ + +int iris_vb2_queue_setup(struct vb2_queue *q, + unsigned int *num_buffers, unsigned int *num_planes, + unsigned int sizes[], struct device *alloc_devs[]); +#endif diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c new file mode 100644 index 000000000000..7d1ef31c7c44 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -0,0 +1,58 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_buffer.h" +#include "iris_instance.h" +#include "iris_vdec.h" +#include "iris_vpu_buffer.h" + +#define DEFAULT_WIDTH 320 +#define DEFAULT_HEIGHT 240 + +void iris_vdec_inst_init(struct iris_inst *inst) +{ + struct v4l2_format *f; + + inst->fmt_src = kzalloc(sizeof(*inst->fmt_src), GFP_KERNEL); + inst->fmt_dst = kzalloc(sizeof(*inst->fmt_dst), GFP_KERNEL); + + inst->fw_min_count = MIN_BUFFERS; + + f = inst->fmt_src; + f->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; + f->fmt.pix_mp.width = DEFAULT_WIDTH; + f->fmt.pix_mp.height = DEFAULT_HEIGHT; + f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_H264; + f->fmt.pix_mp.num_planes = 1; + f->fmt.pix_mp.plane_fmt[0].bytesperline = 0; + f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_INPUT); + f->fmt.pix_mp.field = V4L2_FIELD_NONE; + inst->buffers[BUF_INPUT].min_count = iris_vpu_buf_count(inst, BUF_INPUT); + inst->buffers[BUF_INPUT].actual_count = inst->buffers[BUF_INPUT].min_count; + inst->buffers[BUF_INPUT].size = f->fmt.pix_mp.plane_fmt[0].sizeimage; + + f = inst->fmt_dst; + f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + f->fmt.pix_mp.pixelformat = V4L2_PIX_FMT_NV12; + f->fmt.pix_mp.width = ALIGN(DEFAULT_WIDTH, 128); + f->fmt.pix_mp.height = ALIGN(DEFAULT_HEIGHT, 32); + f->fmt.pix_mp.num_planes = 1; + f->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(DEFAULT_WIDTH, 128); + f->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT); + f->fmt.pix_mp.field = V4L2_FIELD_NONE; + f->fmt.pix_mp.colorspace = V4L2_COLORSPACE_DEFAULT; + f->fmt.pix_mp.xfer_func = V4L2_XFER_FUNC_DEFAULT; + f->fmt.pix_mp.ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + f->fmt.pix_mp.quantization = V4L2_QUANTIZATION_DEFAULT; + inst->buffers[BUF_OUTPUT].min_count = iris_vpu_buf_count(inst, BUF_OUTPUT); + inst->buffers[BUF_OUTPUT].actual_count = inst->buffers[BUF_OUTPUT].min_count; + inst->buffers[BUF_OUTPUT].size = f->fmt.pix_mp.plane_fmt[0].sizeimage; +} + +void iris_vdec_inst_deinit(struct iris_inst *inst) +{ + kfree(inst->fmt_dst); + kfree(inst->fmt_src); +} diff --git a/drivers/media/platform/qcom/iris/iris_vdec.h b/drivers/media/platform/qcom/iris/iris_vdec.h new file mode 100644 index 000000000000..0324d7f796dd --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_vdec.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_VDEC_H_ +#define _IRIS_VDEC_H_ + +struct iris_inst; + +void iris_vdec_inst_init(struct iris_inst *inst); +void iris_vdec_inst_deinit(struct iris_inst *inst); + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index bc0a3232214b..70ce2296d1b2 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -9,6 +9,8 @@ #include "iris_vidc.h" #include "iris_instance.h" +#include "iris_vdec.h" +#include "iris_vb2.h" #include "iris_platform_common.h" #define IRIS_DRV_NAME "iris_driver" @@ -39,6 +41,46 @@ static void iris_v4l2_fh_deinit(struct iris_inst *inst) v4l2_fh_exit(&inst->fh); } +static int iris_add_session(struct iris_inst *inst) +{ + struct iris_core *core = inst->core; + struct iris_inst *i; + u32 count = 0; + int ret = 0; + + mutex_lock(&core->lock); + if (core->state != IRIS_CORE_INIT) { + ret = -EINVAL; + goto unlock; + } + list_for_each_entry(i, &core->instances, list) + count++; + + if (count < core->iris_platform_data->max_session_count) + list_add_tail(&inst->list, &core->instances); + else + ret = -EAGAIN; +unlock: + mutex_unlock(&core->lock); + + return ret; +} + +static void iris_remove_session(struct iris_inst *inst) +{ + struct iris_core *core = inst->core; + struct iris_inst *i, *temp; + + mutex_lock(&core->lock); + list_for_each_entry_safe(i, temp, &core->instances, list) { + if (i->session_id == inst->session_id) { + list_del_init(&i->list); + break; + } + } + mutex_unlock(&core->lock); +} + static struct iris_inst *iris_get_inst(struct file *filp, void *fh) { if (!filp || !filp->private_data) @@ -74,7 +116,9 @@ iris_m2m_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_ src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; src_vq->io_modes = VB2_MMAP | VB2_DMABUF; src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + src_vq->ops = inst->core->iris_vb2_ops; src_vq->drv_priv = inst; + src_vq->buf_struct_size = sizeof(struct iris_buffer); src_vq->dev = inst->core->dev; src_vq->lock = &inst->ctx_q_lock; ret = vb2_queue_init(src_vq); @@ -84,7 +128,9 @@ iris_m2m_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_ dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; dst_vq->io_modes = VB2_MMAP | VB2_DMABUF; dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + dst_vq->ops = inst->core->iris_vb2_ops; dst_vq->drv_priv = inst; + dst_vq->buf_struct_size = sizeof(struct iris_buffer); dst_vq->dev = inst->core->dev; dst_vq->lock = &inst->ctx_q_lock; @@ -115,9 +161,11 @@ int iris_open(struct file *filp) return -ENOMEM; inst->core = core; + inst->session_id = hash32_ptr(inst); mutex_init(&inst->lock); mutex_init(&inst->ctx_q_lock); + init_completion(&inst->completion); ret = iris_v4l2_fh_init(inst); if (ret) @@ -135,11 +183,24 @@ int iris_open(struct file *filp) goto fail_m2m_release; } + iris_vdec_inst_init(inst); + if (ret) + goto fail_m2m_ctx_release; + + ret = iris_add_session(inst); + if (ret) + goto fail_inst_deinit; + inst->fh.m2m_ctx = inst->m2m_ctx; filp->private_data = &inst->fh; return 0; +fail_inst_deinit: + iris_remove_session(inst); + iris_vdec_inst_deinit(inst); +fail_m2m_ctx_release: + v4l2_m2m_ctx_release(inst->m2m_ctx); fail_m2m_release: v4l2_m2m_release(inst->m2m_dev); fail_v4l2_fh_deinit: @@ -152,6 +213,24 @@ int iris_open(struct file *filp) return ret; } +static void iris_session_close(struct iris_inst *inst) +{ + const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + bool wait_for_response; + int ret; + + wait_for_response = true; + + reinit_completion(&inst->completion); + + ret = hfi_ops->session_close(inst); + if (ret) + wait_for_response = false; + + if (wait_for_response) + iris_wait_for_session_response(inst); +} + int iris_close(struct file *filp) { struct iris_inst *inst; @@ -163,7 +242,10 @@ int iris_close(struct file *filp) v4l2_m2m_ctx_release(inst->m2m_ctx); v4l2_m2m_release(inst->m2m_dev); mutex_lock(&inst->lock); + iris_vdec_inst_deinit(inst); + iris_session_close(inst); iris_v4l2_fh_deinit(inst); + iris_remove_session(inst); mutex_unlock(&inst->lock); mutex_destroy(&inst->ctx_q_lock); mutex_destroy(&inst->lock); @@ -182,7 +264,17 @@ static struct v4l2_file_operations iris_v4l2_file_ops = { .mmap = v4l2_m2m_fop_mmap, }; +static const struct vb2_ops iris_vb2_ops = { + .queue_setup = iris_vb2_queue_setup, +}; + +static const struct v4l2_ioctl_ops iris_v4l2_ioctl_ops = { + .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs, +}; + void iris_init_ops(struct iris_core *core) { core->iris_v4l2_file_ops = &iris_v4l2_file_ops; + core->iris_vb2_ops = &iris_vb2_ops; + core->iris_v4l2_ioctl_ops = &iris_v4l2_ioctl_ops; } diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.c b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c new file mode 100644 index 000000000000..2402a33723ab --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c @@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_instance.h" +#include "iris_vpu_buffer.h" + +int iris_vpu_buf_count(struct iris_inst *inst, enum iris_buffer_type buffer_type) +{ + switch (buffer_type) { + case BUF_INPUT: + return MIN_BUFFERS; + case BUF_OUTPUT: + return inst->fw_min_count; + default: + return 0; + } +} diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h new file mode 100644 index 000000000000..f0f974cebd8a --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_VPU_BUFFER_H_ +#define _IRIS_VPU_BUFFER_H_ + +struct iris_inst; + +#define MIN_BUFFERS 4 + +int iris_vpu_buf_count(struct iris_inst *inst, enum iris_buffer_type buffer_type); + +#endif From patchwork Tue Aug 27 10:05:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 822724 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 89DE919F471; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; cv=none; b=ixMlPwe/PrFxQgX8o1r+6grX2eIQm4591ElvccfBxCQ8act6Dr8/5pgdTgCZkg6hQjHs2Jfam63b40Z927VM7d1X/QcXawv9FdR4hjgIgQpRVmvTqdBa5vgqFrXy5Mt63tG9ycdnSr7O2coqmRKROtBKn+1Adtl0bLhAm8uf93k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; c=relaxed/simple; bh=oF/mWsnweX9oZO73H2Wf0/ZJBGASDI30qWA5d2dx3x0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=LJlFHFTs/MMeVA9XM157MBQo6eww6xBO6Tdl3qAfE3v4hn1XirnC5kN0NV1R/HRvxNUHWwYXM8fNoXCyHICWFt0YoAPNleTdpljCrwBHQHRwMsMoRS7fNn7M+xpawihPmybEMtN1LIdD4yJ27CI5McKGrPb48LV/Xt/DTa3SrwQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=oDjOkpEm; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="oDjOkpEm" Received: by smtp.kernel.org (Postfix) with ESMTPS id 501D5C4DE03; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753195; bh=oF/mWsnweX9oZO73H2Wf0/ZJBGASDI30qWA5d2dx3x0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=oDjOkpEmGZWMRs08CBww1jJGsiKT1rnaja/lA56ticy7hQ1jKifd26JeqTlqwB9Pa FBJeZ5XldI+9DeANxvEcltei+W51N9xb01U2yd7cHKZF49s5zTgrtSOhmrsw/jNyKK KPQ9hTcKA49UrT0QLEe42YwxzL4F18qbn4VPZLWqyUA7pFOjLpsjv4OykjxDVSMfKs RinRr9OHmHhFKp41+A/y+p693EsX3w0YPwUtoXJeDhpsJoxC6FGYJKurSI1amioPJC lps9jU7C/C33LtLkpqdpOl1XrTtHL5tJA5vdIAF9PKm4Kz0OJ6uo33FXVP1Ay/zzSh R2X/xvYjNEMbw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3E759C54734; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:37 +0530 Subject: [PATCH v3 12/29] media: iris: implement s_fmt, g_fmt and try_fmt ioctls Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-12-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal , Vedang Nagar X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=8510; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=cGVUlFII0dOQ3M9tJ/hYrs0FBIR1XS40u8Ojj4Oi09s=; b=k8khEoO5onw/ea1JITGQVKrc0uzdvVbFDhOTbZhJyXVqTPoqliHZp9/DG1UTO9ToiA+Duguq0 wvOoKCyZnAJBADrP5B0EPGm8uH9AOqTLDA3tvPhnPYysJ7FqOg65HD5 X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Vedang Nagar Implement s_fmt, g_fmt and try_fmt IOCTL ops in the driver with necessary hooks. Signed-off-by: Vedang Nagar Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/iris_vdec.c | 129 +++++++++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_vdec.h | 2 + drivers/media/platform/qcom/iris/iris_vidc.c | 60 +++++++++++++ 3 files changed, 191 insertions(+) diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index 7d1ef31c7c44..fd04f56b405d 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -3,6 +3,8 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include + #include "iris_buffer.h" #include "iris_instance.h" #include "iris_vdec.h" @@ -10,6 +12,7 @@ #define DEFAULT_WIDTH 320 #define DEFAULT_HEIGHT 240 +#define DEFAULT_CODEC_ALIGNMENT 16 void iris_vdec_inst_init(struct iris_inst *inst) { @@ -56,3 +59,129 @@ void iris_vdec_inst_deinit(struct iris_inst *inst) kfree(inst->fmt_dst); kfree(inst->fmt_src); } + +int iris_vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f) +{ + struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp; + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + struct v4l2_format *f_inst; + struct vb2_queue *src_q; + + src_q = v4l2_m2m_get_src_vq(m2m_ctx); + memset(pixmp->reserved, 0, sizeof(pixmp->reserved)); + if (V4L2_TYPE_IS_OUTPUT(f->type)) { + if (f->fmt.pix_mp.pixelformat != V4L2_PIX_FMT_H264) { + f_inst = inst->fmt_src; + f->fmt.pix_mp.width = f_inst->fmt.pix_mp.width; + f->fmt.pix_mp.height = f_inst->fmt.pix_mp.height; + f->fmt.pix_mp.pixelformat = f_inst->fmt.pix_mp.pixelformat; + } + } else if (V4L2_TYPE_IS_CAPTURE(f->type)) { + if (f->fmt.pix_mp.pixelformat != V4L2_PIX_FMT_NV12) { + f_inst = inst->fmt_dst; + f->fmt.pix_mp.pixelformat = f_inst->fmt.pix_mp.pixelformat; + f->fmt.pix_mp.width = f_inst->fmt.pix_mp.width; + f->fmt.pix_mp.height = f_inst->fmt.pix_mp.height; + } + if (vb2_is_streaming(src_q)) { + f_inst = inst->fmt_src; + f->fmt.pix_mp.height = f_inst->fmt.pix_mp.height; + f->fmt.pix_mp.width = f_inst->fmt.pix_mp.width; + } + } else { + return -EINVAL; + } + + if (pixmp->field == V4L2_FIELD_ANY) + pixmp->field = V4L2_FIELD_NONE; + + pixmp->num_planes = 1; + + return 0; +} + +int iris_vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f) +{ + struct v4l2_format *fmt, *output_fmt; + struct vb2_queue *q; + u32 codec_align; + + int ret = 0; + + q = v4l2_m2m_get_vq(inst->m2m_ctx, f->type); + if (!q) + return -EINVAL; + + iris_vdec_try_fmt(inst, f); + + if (V4L2_TYPE_IS_OUTPUT(f->type)) { + if (f->fmt.pix_mp.pixelformat != V4L2_PIX_FMT_H264) + return -EINVAL; + + fmt = inst->fmt_src; + fmt->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; + + codec_align = DEFAULT_CODEC_ALIGNMENT; + fmt->fmt.pix_mp.width = ALIGN(f->fmt.pix_mp.width, codec_align); + fmt->fmt.pix_mp.height = ALIGN(f->fmt.pix_mp.height, codec_align); + fmt->fmt.pix_mp.num_planes = 1; + fmt->fmt.pix_mp.plane_fmt[0].bytesperline = 0; + fmt->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_INPUT); + inst->buffers[BUF_INPUT].min_count = iris_vpu_buf_count(inst, BUF_INPUT); + if (inst->buffers[BUF_INPUT].actual_count < inst->buffers[BUF_INPUT].min_count) + inst->buffers[BUF_INPUT].actual_count = inst->buffers[BUF_INPUT].min_count; + + inst->buffers[BUF_INPUT].size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage; + + fmt->fmt.pix_mp.colorspace = f->fmt.pix_mp.colorspace; + fmt->fmt.pix_mp.xfer_func = f->fmt.pix_mp.xfer_func; + fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc; + fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization; + + output_fmt = inst->fmt_dst; + output_fmt->fmt.pix_mp.colorspace = f->fmt.pix_mp.colorspace; + output_fmt->fmt.pix_mp.xfer_func = f->fmt.pix_mp.xfer_func; + output_fmt->fmt.pix_mp.ycbcr_enc = f->fmt.pix_mp.ycbcr_enc; + output_fmt->fmt.pix_mp.quantization = f->fmt.pix_mp.quantization; + + inst->crop.left = 0; + inst->crop.top = 0; + inst->crop.width = f->fmt.pix_mp.width; + inst->crop.height = f->fmt.pix_mp.height; + } else if (V4L2_TYPE_IS_CAPTURE(f->type)) { + fmt = inst->fmt_dst; + fmt->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + if (q->streaming) { + f->fmt.pix_mp.height = inst->fmt_src->fmt.pix_mp.height; + f->fmt.pix_mp.width = inst->fmt_src->fmt.pix_mp.width; + } + if (fmt->fmt.pix_mp.pixelformat != V4L2_PIX_FMT_NV12) + return -EINVAL; + fmt->fmt.pix_mp.pixelformat = f->fmt.pix_mp.pixelformat; + fmt->fmt.pix_mp.width = ALIGN(f->fmt.pix_mp.width, 128); + fmt->fmt.pix_mp.height = ALIGN(f->fmt.pix_mp.height, 32); + fmt->fmt.pix_mp.num_planes = 1; + fmt->fmt.pix_mp.plane_fmt[0].bytesperline = ALIGN(f->fmt.pix_mp.width, 128); + fmt->fmt.pix_mp.plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT); + + if (!q->streaming) + inst->buffers[BUF_OUTPUT].min_count = iris_vpu_buf_count(inst, BUF_INPUT); + if (inst->buffers[BUF_OUTPUT].actual_count < inst->buffers[BUF_OUTPUT].min_count) + inst->buffers[BUF_OUTPUT].actual_count = + inst->buffers[BUF_OUTPUT].min_count; + + inst->buffers[BUF_OUTPUT].size = fmt->fmt.pix_mp.plane_fmt[0].sizeimage; + + if (!q->streaming) { + inst->crop.top = 0; + inst->crop.left = 0; + inst->crop.width = f->fmt.pix_mp.width; + inst->crop.height = f->fmt.pix_mp.height; + } + } else { + return -EINVAL; + } + memcpy(f, fmt, sizeof(*fmt)); + + return ret; +} diff --git a/drivers/media/platform/qcom/iris/iris_vdec.h b/drivers/media/platform/qcom/iris/iris_vdec.h index 0324d7f796dd..4f2557d15ca2 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.h +++ b/drivers/media/platform/qcom/iris/iris_vdec.h @@ -10,5 +10,7 @@ struct iris_inst; void iris_vdec_inst_init(struct iris_inst *inst); void iris_vdec_inst_deinit(struct iris_inst *inst); +int iris_vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f); +int iris_vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f); #endif diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index 70ce2296d1b2..7eafbbfd22e4 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -255,6 +255,60 @@ int iris_close(struct file *filp) return 0; } +static int iris_try_fmt_vid_mplane(struct file *filp, void *fh, struct v4l2_format *f) +{ + struct iris_inst *inst; + int ret; + + inst = iris_get_inst(filp, fh); + if (!inst) + return -EINVAL; + + mutex_lock(&inst->lock); + ret = iris_vdec_try_fmt(inst, f); + mutex_unlock(&inst->lock); + + return ret; +} + +static int iris_s_fmt_vid_mplane(struct file *filp, void *fh, struct v4l2_format *f) +{ + struct iris_inst *inst; + int ret; + + inst = iris_get_inst(filp, fh); + if (!inst) + return -EINVAL; + + mutex_lock(&inst->lock); + ret = iris_vdec_s_fmt(inst, f); + mutex_unlock(&inst->lock); + + return ret; +} + +static int iris_g_fmt_vid_mplane(struct file *filp, void *fh, struct v4l2_format *f) +{ + struct iris_inst *inst; + int ret = 0; + + inst = iris_get_inst(filp, fh); + if (!inst) + return -EINVAL; + + mutex_lock(&inst->lock); + if (V4L2_TYPE_IS_OUTPUT(f->type)) + memcpy(f, inst->fmt_src, sizeof(*f)); + else if (V4L2_TYPE_IS_CAPTURE(f->type)) + memcpy(f, inst->fmt_dst, sizeof(*f)); + else + ret = -EINVAL; + + mutex_unlock(&inst->lock); + + return ret; +} + static struct v4l2_file_operations iris_v4l2_file_ops = { .owner = THIS_MODULE, .open = iris_open, @@ -269,6 +323,12 @@ static const struct vb2_ops iris_vb2_ops = { }; static const struct v4l2_ioctl_ops iris_v4l2_ioctl_ops = { + .vidioc_try_fmt_vid_cap_mplane = iris_try_fmt_vid_mplane, + .vidioc_try_fmt_vid_out_mplane = iris_try_fmt_vid_mplane, + .vidioc_s_fmt_vid_cap_mplane = iris_s_fmt_vid_mplane, + .vidioc_s_fmt_vid_out_mplane = iris_s_fmt_vid_mplane, + .vidioc_g_fmt_vid_cap_mplane = iris_g_fmt_vid_mplane, + .vidioc_g_fmt_vid_out_mplane = iris_g_fmt_vid_mplane, .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs, }; From patchwork Tue Aug 27 10:05:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 824189 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A642C19F49A; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; cv=none; b=t/gnbCQPu3RkB8ksibtbvvg+GtzfrzxegxxYgOXxwRBmP5qyVqsPX0tCHaOk+Qghlmx2zE6uQaz/oM6DosQjfZV/EigGxbqe/iWWpJQT5fGZPVrxVJ+BMOFqqzvGW2OG5bdlUjmhdpXAvf62LwgM69lK0hZtYbiAWeweLI2QE1Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; c=relaxed/simple; bh=qkHyU9JW9UhkdBZh1qj7g9q6Ks9zEJX5pMeK7C7N1a8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=lcJvTQCtJJ5YQFcVu1Vlv8lBZKoA7ZEgCutn/Hto0lzv77ip2UuJ78etqCPTiHuHdv/2pbeC3QMBJiqW6Vmp6f6rjlZ6qj9OJ5KXtKLsffRH3cHmtQ3Wxqc2hBf+VjR1PNHb20/wbjGGVa372wcNHKxU3QDf9/WeRj531CAK+o4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QtqI8xtq; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QtqI8xtq" Received: by smtp.kernel.org (Postfix) with ESMTPS id 6CD28C4DE1B; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753195; bh=qkHyU9JW9UhkdBZh1qj7g9q6Ks9zEJX5pMeK7C7N1a8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=QtqI8xtqPtlrRZAW7bzkKZbU/Gf9wz4J4T9UNTMoFGte0d6uhJeAeETKVGku4+fcn 5RiEEm18k/u8LAM4KyDuy12jiYueLwpCYPIfbOy1iAAoDriA5IKcnK/B9M4gKnxDMe spgGC37Vf6ibdZh8OQeq9c0h9gtspn28uZJgZMOEmUTZH+U7wCPhu1TjHKlLGJ0Rxf c2a9XPpeZScJqsLQ1pMrqyTDbmUb4THy2NHk5dS6QrWRJScMWX9GeIHn4brFkLD8BR 8SEqhi7hyZBaQsUqmX6Ac/JcbvksUI9sPnysJyDSP99rsW2uEEU0TxBrXMFy3L4Xso uD6xJaj9m30jw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59D6DC52D6F; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:38 +0530 Subject: [PATCH v3 13/29] media: iris: implement g_selection ioctl Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-13-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal , Vedang Nagar X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=2127; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=p8lMxAQLIngNvbWVVTC34P+pqjWzrRVYcTab/Vl0ejY=; b=agEiI+HjFJi1eDip1iEXGl/3HSu6vIZykLsuSzjw9QBOLa1sg4rMAsEI+pEHAd72iSHnsCs9l +ePdAPDkFIMDIW5M2wLBeFfLeuukTmrsZ1Wxfl0BsGhz8ORg7dd30zS X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Vedang Nagar Implement g_selection ioctl in the driver with necessary hooks. Signed-off-by: Vedang Nagar Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/iris_vidc.c | 40 ++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index 7eafbbfd22e4..e09c0a156387 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -309,6 +309,45 @@ static int iris_g_fmt_vid_mplane(struct file *filp, void *fh, struct v4l2_format return ret; } +static int iris_g_selection(struct file *filp, void *fh, struct v4l2_selection *s) +{ + struct iris_inst *inst; + int ret = 0; + + inst = iris_get_inst(filp, fh); + if (!inst || !s) + return -EINVAL; + + mutex_lock(&inst->lock); + if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE && + s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE) { + ret = -EINVAL; + goto unlock; + } + + switch (s->target) { + case V4L2_SEL_TGT_CROP_BOUNDS: + case V4L2_SEL_TGT_CROP_DEFAULT: + case V4L2_SEL_TGT_CROP: + case V4L2_SEL_TGT_COMPOSE_BOUNDS: + case V4L2_SEL_TGT_COMPOSE_PADDED: + case V4L2_SEL_TGT_COMPOSE_DEFAULT: + case V4L2_SEL_TGT_COMPOSE: + s->r.left = inst->crop.left; + s->r.top = inst->crop.top; + s->r.width = inst->crop.width; + s->r.height = inst->crop.height; + break; + default: + ret = -EINVAL; + } + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + static struct v4l2_file_operations iris_v4l2_file_ops = { .owner = THIS_MODULE, .open = iris_open, @@ -330,6 +369,7 @@ static const struct v4l2_ioctl_ops iris_v4l2_ioctl_ops = { .vidioc_g_fmt_vid_cap_mplane = iris_g_fmt_vid_mplane, .vidioc_g_fmt_vid_out_mplane = iris_g_fmt_vid_mplane, .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs, + .vidioc_g_selection = iris_g_selection, }; void iris_init_ops(struct iris_core *core) From patchwork Tue Aug 27 10:05:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 824190 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A63A819F499; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; cv=none; b=tQoNyaaS8a8XLCPjAroMHitfJNBQ/t2hoxEHrq5l/faNxaGbTQSjv8Ml9xro7H8mhPB6xw5TxtSHg8YEyLqrOMQzcC1h8/q9nL57b9up0cU59apGzF1BArQS/nXA1zF5sC8NnFSbyq7HsLbK3bNE6qG/MOjNLqtsUOIg6exjg3o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; c=relaxed/simple; bh=x7sN4vAGNoMnwJ1OWB1fVULGlt5glIJiJZmZWurTS3A=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=qHe08o3hc/n/67kDUSiIDVFMcyed1SptKcl8zFC/f0ev6pdf+SGpXFTSwxK3Z/BwIN+VBXmURsBsbfK5BlZDmK+ASxWrp64B9JiH4GSXXKfJwJ3yVrR3mTLYCe2FwANttwgIMWeab3RjINGRt+KJTOnVS7sOzysWMFH8dKyNDms= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=e1mJU63/; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="e1mJU63/" Received: by smtp.kernel.org (Postfix) with ESMTPS id 7B8E1C4DE1E; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753195; bh=x7sN4vAGNoMnwJ1OWB1fVULGlt5glIJiJZmZWurTS3A=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=e1mJU63/r4tThYvq6Q/0DdYx+O4c5fQj2YUoI7Py2Il+9oIWoeoN6pgY3sKlFGcKA l5SyYBqM6W5RMDunJ3etAetmEfEAXsU2SogdWjMfnGddp60MmEw9ugXA0i7QsxfYHG czyIXPLsz64Ypc8ZZwVRF3Z2yMYdlCXoINaTIdIoWqYeHhup0FH4956HNx+0Q0mrHV zLx5Cjo3U/p5cD3OUo4UA/EZxiGKwkblycqKtke6kzEk1qamKuvGyyLFGpadzzNTsz EKlFuht+bFb1zhZtz9znoEfM3M1AfuYFF6NW8GhM9RC+PaiEfMSRGcGLwFaRfQ27xn QeWh7+QBAbetw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7141AC54734; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:39 +0530 Subject: [PATCH v3 14/29] media: iris: implement enum_fmt and enum_frameintervals ioctls Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-14-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal , Vedang Nagar X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=5098; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=QCr6J53yyQB3qsIP/vgwzzl2dZykbmOi6iYbyLA24lI=; b=TM067bJniiwaBsNPc31rJbmpH4nB/LQsof15R0x7qq8i+yTXWjXGv6TSDB0ToyTSXfy+qkF3W uYUS1WzehRKDywV3eTS2T3GoIRoF4gBOmO7UR3J4AcUSENsDOZTIgDj X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Vedang Nagar Implement enum_fmt and enum_frameintervals ioctls in the driver with necessary hooks. Signed-off-by: Vedang Nagar Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/iris_vdec.c | 22 +++++++++++ drivers/media/platform/qcom/iris/iris_vdec.h | 1 + drivers/media/platform/qcom/iris/iris_vidc.c | 57 ++++++++++++++++++++++++++++ 3 files changed, 80 insertions(+) diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index fd04f56b405d..a38903eefa1c 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -60,6 +60,28 @@ void iris_vdec_inst_deinit(struct iris_inst *inst) kfree(inst->fmt_src); } +int iris_vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f) +{ + if (f->index) + return -EINVAL; + + if (V4L2_TYPE_IS_OUTPUT(f->type)) { + f->pixelformat = V4L2_PIX_FMT_H264; + f->flags = V4L2_FMT_FLAG_COMPRESSED | V4L2_FMT_FLAG_DYN_RESOLUTION; + strscpy(f->description, "codec", sizeof(f->description)); + } else if (V4L2_TYPE_IS_CAPTURE(f->type)) { + f->pixelformat = V4L2_PIX_FMT_NV12; + strscpy(f->description, "colorformat", sizeof(f->description)); + } + + if (!f->pixelformat) + return -EINVAL; + + memset(f->reserved, 0, sizeof(f->reserved)); + + return 0; +} + int iris_vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f) { struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp; diff --git a/drivers/media/platform/qcom/iris/iris_vdec.h b/drivers/media/platform/qcom/iris/iris_vdec.h index 4f2557d15ca2..eb8a1121ae92 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.h +++ b/drivers/media/platform/qcom/iris/iris_vdec.h @@ -10,6 +10,7 @@ struct iris_inst; void iris_vdec_inst_init(struct iris_inst *inst); void iris_vdec_inst_deinit(struct iris_inst *inst); +int iris_vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f); int iris_vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f); int iris_vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f); diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index e09c0a156387..488be6afd999 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -15,6 +15,10 @@ #define IRIS_DRV_NAME "iris_driver" #define IRIS_BUS_NAME "platform:iris_icc" +#define MIN_FRAME_WIDTH 128 +#define MAX_FRAME_WIDTH 8192 +#define MIN_FRAME_HEIGHT 128 +#define MAX_FRAME_HEIGHT 8192 #define STEP_WIDTH 1 #define STEP_HEIGHT 1 @@ -255,6 +259,22 @@ int iris_close(struct file *filp) return 0; } +static int iris_enum_fmt(struct file *filp, void *fh, struct v4l2_fmtdesc *f) +{ + struct iris_inst *inst; + int ret; + + inst = iris_get_inst(filp, fh); + if (!inst) + return -EINVAL; + + mutex_lock(&inst->lock); + ret = iris_vdec_enum_fmt(inst, f); + mutex_unlock(&inst->lock); + + return ret; +} + static int iris_try_fmt_vid_mplane(struct file *filp, void *fh, struct v4l2_format *f) { struct iris_inst *inst; @@ -309,6 +329,40 @@ static int iris_g_fmt_vid_mplane(struct file *filp, void *fh, struct v4l2_format return ret; } +static int iris_enum_framesizes(struct file *filp, void *fh, + struct v4l2_frmsizeenum *fsize) +{ + struct iris_inst *inst; + int ret = 0; + + inst = iris_get_inst(filp, fh); + if (!inst || !fsize) + return -EINVAL; + + if (fsize->index) + return -EINVAL; + + mutex_lock(&inst->lock); + if (fsize->pixel_format != V4L2_PIX_FMT_H264 && + fsize->pixel_format != V4L2_PIX_FMT_NV12) { + ret = -EINVAL; + goto unlock; + } + + fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE; + fsize->stepwise.min_width = MIN_FRAME_WIDTH; + fsize->stepwise.max_width = MAX_FRAME_WIDTH; + fsize->stepwise.step_width = STEP_WIDTH; + fsize->stepwise.min_height = MIN_FRAME_HEIGHT; + fsize->stepwise.max_height = MAX_FRAME_HEIGHT; + fsize->stepwise.step_height = STEP_HEIGHT; + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + static int iris_g_selection(struct file *filp, void *fh, struct v4l2_selection *s) { struct iris_inst *inst; @@ -362,12 +416,15 @@ static const struct vb2_ops iris_vb2_ops = { }; static const struct v4l2_ioctl_ops iris_v4l2_ioctl_ops = { + .vidioc_enum_fmt_vid_cap = iris_enum_fmt, + .vidioc_enum_fmt_vid_out = iris_enum_fmt, .vidioc_try_fmt_vid_cap_mplane = iris_try_fmt_vid_mplane, .vidioc_try_fmt_vid_out_mplane = iris_try_fmt_vid_mplane, .vidioc_s_fmt_vid_cap_mplane = iris_s_fmt_vid_mplane, .vidioc_s_fmt_vid_out_mplane = iris_s_fmt_vid_mplane, .vidioc_g_fmt_vid_cap_mplane = iris_g_fmt_vid_mplane, .vidioc_g_fmt_vid_out_mplane = iris_g_fmt_vid_mplane, + .vidioc_enum_framesizes = iris_enum_framesizes, .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs, .vidioc_g_selection = iris_g_selection, }; From patchwork Tue Aug 27 10:05:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 822723 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C001419FA86; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; cv=none; b=On9omUnY0sdkGXWutXjkUvpkGO/n9GkQNkEhovdYvd32nnVriqnhFdKwoznFx4ZX9G8a6s7iIY7YtJuRK2Gjl1KCsB9OsPWm/imQ1SRkAm7YgymFRTl/qgWkPGU7Th05uH8THOZnJgtUVgFd740H6sNeWGwITF/gqNZT4nLRe74= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753195; c=relaxed/simple; bh=r9jUpb5sjdrgtqpcrmScsEF39c7IVq/pH9K5zGUswTg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=LJhMPF7YVQAQuBUgFOVQIHXh2GWuXx7GoQGxUIimf55Ib5CVYz9DvtoG4V7C0Ur+kBQz1FTU9MkqtrVI2zAifUdyc4lF0iF1wN/EBsjnqNVWB5GEFA1JAwEU0Qp/8d4tEUQvwTZvurJkLJnCmjLERdAeX6kyRAAcFcL3GfaU5C0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=riNkK4GJ; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="riNkK4GJ" Received: by smtp.kernel.org (Postfix) with ESMTPS id 9AFA9C4AF50; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753195; bh=r9jUpb5sjdrgtqpcrmScsEF39c7IVq/pH9K5zGUswTg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=riNkK4GJUB05XV8c5TosTpMpQRg38jrduiyk6N8ROFYGNhH0YrZQbHkxyeMSvizEt jTKQQOD4SlgM3plU2OEliHCNEsNhPNsNyF0oP5NK/59lJ7WM9JnEzDBha9rLugD2Vh R2lyYbnCCpK+0LKuTzCngopC+OWK7Tx8za2v+0Cygp5g0qM4ZJgFAY/dKfYhdqcclb swnyq9tgq7OiyxFH67KtMfZPhbt5Zn8k9ospbCU5Gnu39bJo44Aad/6vqVh/er2Ft6 4kOY4j6TEAegArPRpmKidvbAs9JvepY17j3WtXqyarvo4baXKSeCvx6MKOFkjcK/JT E/8c1qbGQ2eXQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 890D5C52D6F; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:40 +0530 Subject: [PATCH v3 15/29] media: iris: implement subscribe_event and unsubscribe_event ioctls Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-15-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal , Vedang Nagar X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=5119; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=nP3/atOpfa9eV16b785gciUAZJmLxBjkVrJb7x2W4qI=; b=IvsM+jlYsuyNinbW6BiUye8VHSkOT4bXRCXhpVeeSnOIreWTP+MzP6kPM4H85NL20vLZzLy4n +T3+8pVKD7pBofEWwQmsgROQpfmcud3oBBd/A0lGt4eE0WVOMTzUmAU X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Vedang Nagar Implement subscribe_event and unsubscribe_event ioctls in the driver with necessary hooks. Signed-off-by: Vedang Nagar Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/iris_instance.h | 2 ++ drivers/media/platform/qcom/iris/iris_vdec.c | 26 ++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_vdec.h | 1 + drivers/media/platform/qcom/iris/iris_vidc.c | 31 ++++++++++++++++++++++++ 4 files changed, 60 insertions(+) diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h index bb43119af352..d28b8fd7ec2f 100644 --- a/drivers/media/platform/qcom/iris/iris_instance.h +++ b/drivers/media/platform/qcom/iris/iris_instance.h @@ -30,6 +30,7 @@ * @once_per_session_set: boolean to set once per session property * @m2m_dev: a reference to m2m device structure * @m2m_ctx: a reference to m2m context structure + * @subscriptions: variable to hold current events subscriptions */ struct iris_inst { @@ -48,6 +49,7 @@ struct iris_inst { bool once_per_session_set; struct v4l2_m2m_dev *m2m_dev; struct v4l2_m2m_ctx *m2m_ctx; + unsigned int subscriptions; }; #endif diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index a38903eefa1c..29c7e229c148 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -3,6 +3,7 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include #include #include "iris_buffer.h" @@ -13,6 +14,7 @@ #define DEFAULT_WIDTH 320 #define DEFAULT_HEIGHT 240 #define DEFAULT_CODEC_ALIGNMENT 16 +#define MAX_EVENTS 30 void iris_vdec_inst_init(struct iris_inst *inst) { @@ -207,3 +209,27 @@ int iris_vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f) return ret; } + +int iris_vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub) +{ + int ret = 0; + + switch (sub->type) { + case V4L2_EVENT_EOS: + ret = v4l2_event_subscribe(&inst->fh, sub, MAX_EVENTS, NULL); + inst->subscriptions |= V4L2_EVENT_EOS; + break; + case V4L2_EVENT_SOURCE_CHANGE: + ret = v4l2_src_change_event_subscribe(&inst->fh, sub); + inst->subscriptions |= V4L2_EVENT_SOURCE_CHANGE; + break; + case V4L2_EVENT_CTRL: + ret = v4l2_ctrl_subscribe_event(&inst->fh, sub); + inst->subscriptions |= V4L2_EVENT_CTRL; + break; + default: + return -EINVAL; + } + + return ret; +} diff --git a/drivers/media/platform/qcom/iris/iris_vdec.h b/drivers/media/platform/qcom/iris/iris_vdec.h index eb8a1121ae92..707fff34bf4d 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.h +++ b/drivers/media/platform/qcom/iris/iris_vdec.h @@ -13,5 +13,6 @@ void iris_vdec_inst_deinit(struct iris_inst *inst); int iris_vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f); int iris_vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f); int iris_vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f); +int iris_vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub); #endif diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index 488be6afd999..141704512b3c 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -4,6 +4,7 @@ */ #include +#include #include #include @@ -402,6 +403,34 @@ static int iris_g_selection(struct file *filp, void *fh, struct v4l2_selection * return ret; } +static int iris_subscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subscription *sub) +{ + struct iris_inst *inst; + int ret; + + inst = container_of(fh, struct iris_inst, fh); + + mutex_lock(&inst->lock); + ret = iris_vdec_subscribe_event(inst, sub); + mutex_unlock(&inst->lock); + + return ret; +} + +static int iris_unsubscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subscription *sub) +{ + struct iris_inst *inst; + int ret; + + inst = container_of(fh, struct iris_inst, fh); + + mutex_lock(&inst->lock); + ret = v4l2_event_unsubscribe(&inst->fh, sub); + mutex_unlock(&inst->lock); + + return ret; +} + static struct v4l2_file_operations iris_v4l2_file_ops = { .owner = THIS_MODULE, .open = iris_open, @@ -427,6 +456,8 @@ static const struct v4l2_ioctl_ops iris_v4l2_ioctl_ops = { .vidioc_enum_framesizes = iris_enum_framesizes, .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs, .vidioc_g_selection = iris_g_selection, + .vidioc_subscribe_event = iris_subscribe_event, + .vidioc_unsubscribe_event = iris_unsubscribe_event, }; void iris_init_ops(struct iris_core *core) From patchwork Tue Aug 27 10:05:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 824186 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D95741A00D3; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; cv=none; b=BqFt2iXmJL259jG0O5nnS8fNLiG5hUasfKt+5dT423ug01Cd5o2uNUUwJFj2lkp1EYJ8TbEgF8O4t/FYeUU55n1yTfUB9N4ZWDgHFJ8Emjw+9x5KZwsGCNUEqol/eH5X5ZFoz0PBNR2ygA4kkpD6fpA+HDEq5+AWJbSAhDYXgB0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; c=relaxed/simple; bh=s7lcqMLMCAvjiOQXF+yUvPPn6c1YKH7prWBkiQcZTvI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=LQOSwOV17wZGgLPGnrwKdvaM7R745a/UDWG3srQUt0l1gdezVz2eUL2Yz5kE7CaRHzyArrxpKdBIMW3ZMttxJ5eAiFwSk1SPmhJ/bcJ7dq44sLalSqTAUkN1wSJqPu4tCJ+qDbvoLJRjdCbvj+rUQkyGMxS/UeIQKckTUwnvdzA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=TUvz4Ui6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="TUvz4Ui6" Received: by smtp.kernel.org (Postfix) with ESMTPS id A9812C4FDE1; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753195; bh=s7lcqMLMCAvjiOQXF+yUvPPn6c1YKH7prWBkiQcZTvI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=TUvz4Ui63GWoY9qCC0kbHTVpUrJ0YxNyGXkAGTz6b/Nx5nBTXLC2Hk9ASOduVNegM HIteax6k9Rioq3cHi7NxRSESW21CDd6RBBbkXQGoRYcvnVxpDqDhaSsjrK//aTXBfr 1pFJ9QS2qX30VLd74sw+pNw/usRq19vjPZ9Jpuj4cLQmOggi0B+uoYlNygj4RP52Rw LB1ppXmOpIDiQq7tj0TdrVTav/AgXTgNE9OGc3U7YFocZWxkzLz1zz0C5wfyIllAg3 wM8EEwShefunyIpx9WpBIMy8QZv/aAkwEWNb6LmIwc8oPP9Y5s88xMg2LqF1386//D R6fl9nqPTBxDA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FDC3C54734; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:41 +0530 Subject: [PATCH v3 16/29] media: iris: implement iris v4l2_ctrl_ops and prepare capabilities Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-16-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal , Vedang Nagar X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=27871; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=mfTBSSp/p0cZpwAn10kCxzzxI8rQoAYUon451UQ4o2o=; b=F8cyKTJE9po4knabE3pwYrtRyyv7zBmaKRdxtfp/GSAK9tU7cH539LGqw3ltMYtep3qRDv3/b ZQ/K8u7L6oTBd1cPIHkTDt/RXtQX0TbR9ftuGw8U9pKA/HEpWpnmBlV X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Vedang Nagar Implement s_ctrl and g_volatile_ctrl ctrl ops. Introduce platform specific driver and firmware capabilities. Capabilities are set of video specifications and features supported by a specific platform (SOC). Each capability is defined with min, max, range, default value and corresponding HFI. Signed-off-by: Vedang Nagar Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/Makefile | 1 + drivers/media/platform/qcom/iris/iris_buffer.c | 3 +- drivers/media/platform/qcom/iris/iris_core.h | 2 + drivers/media/platform/qcom/iris/iris_ctrls.c | 194 +++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_ctrls.h | 15 ++ .../platform/qcom/iris/iris_hfi_gen1_defines.h | 4 + .../platform/qcom/iris/iris_hfi_gen2_command.c | 1 + .../platform/qcom/iris/iris_hfi_gen2_defines.h | 9 + drivers/media/platform/qcom/iris/iris_instance.h | 6 + .../platform/qcom/iris/iris_platform_common.h | 71 ++++++++ .../platform/qcom/iris/iris_platform_sm8250.c | 56 ++++++ .../platform/qcom/iris/iris_platform_sm8550.c | 138 +++++++++++++++ drivers/media/platform/qcom/iris/iris_probe.c | 7 + drivers/media/platform/qcom/iris/iris_vdec.c | 24 ++- drivers/media/platform/qcom/iris/iris_vdec.h | 2 +- drivers/media/platform/qcom/iris/iris_vidc.c | 16 +- 16 files changed, 536 insertions(+), 13 deletions(-) diff --git a/drivers/media/platform/qcom/iris/Makefile b/drivers/media/platform/qcom/iris/Makefile index 9c50e29db41e..a746681e03cd 100644 --- a/drivers/media/platform/qcom/iris/Makefile +++ b/drivers/media/platform/qcom/iris/Makefile @@ -1,5 +1,6 @@ iris-objs += iris_buffer.o \ iris_core.o \ + iris_ctrls.o \ iris_firmware.o \ iris_hfi_common.o \ iris_hfi_gen1_command.o \ diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c b/drivers/media/platform/qcom/iris/iris_buffer.c index a1017ceede7d..652117a19b45 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/iris/iris_buffer.c @@ -12,7 +12,6 @@ #define MAX_WIDTH 4096 #define MAX_HEIGHT 2304 #define NUM_MBS_4K (DIV_ROUND_UP(MAX_WIDTH, 16) * DIV_ROUND_UP(MAX_HEIGHT, 16)) -#define BASE_RES_MB_MAX 138240 /* * NV12: @@ -74,7 +73,7 @@ static u32 iris_input_buffer_size(struct iris_inst *inst) num_mbs = iris_get_mbpf(inst); if (num_mbs > NUM_MBS_4K) { div_factor = 4; - base_res_mbs = BASE_RES_MB_MAX; + base_res_mbs = inst->driver_cap[MBPF].value; } else { base_res_mbs = NUM_MBS_4K; div_factor = 2; diff --git a/drivers/media/platform/qcom/iris/iris_core.h b/drivers/media/platform/qcom/iris/iris_core.h index 1f6eca31928d..657d26a0fa2e 100644 --- a/drivers/media/platform/qcom/iris/iris_core.h +++ b/drivers/media/platform/qcom/iris/iris_core.h @@ -58,6 +58,7 @@ * @intr_status: interrupt status * @sys_error_handler: a delayed work for handling system fatal error * @instances: a list_head of all instances + * @inst_fw_cap: an array of supported instance capabilities */ struct iris_core { @@ -97,6 +98,7 @@ struct iris_core { u32 intr_status; struct delayed_work sys_error_handler; struct list_head instances; + struct platform_inst_fw_cap inst_fw_cap[INST_FW_CAP_MAX]; }; int iris_core_init(struct iris_core *core); diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.c b/drivers/media/platform/qcom/iris/iris_ctrls.c new file mode 100644 index 000000000000..868306d68a87 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_ctrls.c @@ -0,0 +1,194 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include "iris_ctrls.h" +#include "iris_instance.h" + +static bool iris_valid_cap_id(enum platform_inst_fw_cap_type cap_id) +{ + return cap_id >= 1 && cap_id < INST_FW_CAP_MAX; +} + +static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id) +{ + switch (id) { + case V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER: + return DEBLOCK; + case V4L2_CID_MPEG_VIDEO_H264_PROFILE: + return PROFILE; + case V4L2_CID_MPEG_VIDEO_H264_LEVEL: + return LEVEL; + default: + return INST_FW_CAP_MAX; + } +} + +static u32 iris_get_v4l2_id(enum platform_inst_fw_cap_type cap_id) +{ + if (!iris_valid_cap_id(cap_id)) + return 0; + + switch (cap_id) { + case DEBLOCK: + return V4L2_CID_MPEG_VIDEO_DECODER_MPEG4_DEBLOCK_FILTER; + case PROFILE: + return V4L2_CID_MPEG_VIDEO_H264_PROFILE; + case LEVEL: + return V4L2_CID_MPEG_VIDEO_H264_LEVEL; + default: + return 0; + } +} + +static int iris_vdec_op_g_volatile_ctrl(struct v4l2_ctrl *ctrl) +{ + enum platform_inst_fw_cap_type cap_id; + struct iris_inst *inst = NULL; + + inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler); + switch (ctrl->id) { + case V4L2_CID_MIN_BUFFERS_FOR_CAPTURE: + ctrl->val = inst->buffers[BUF_OUTPUT].min_count; + break; + case V4L2_CID_MIN_BUFFERS_FOR_OUTPUT: + ctrl->val = inst->buffers[BUF_INPUT].min_count; + break; + default: + cap_id = iris_get_cap_id(ctrl->id); + if (iris_valid_cap_id(cap_id)) + ctrl->val = inst->fw_cap[cap_id].value; + else + return -EINVAL; + } + + return 0; +} + +static int iris_vdec_op_s_ctrl(struct v4l2_ctrl *ctrl) +{ + enum platform_inst_fw_cap_type cap_id; + struct platform_inst_fw_cap *cap; + struct iris_inst *inst; + + inst = container_of(ctrl->handler, struct iris_inst, ctrl_handler); + cap = &inst->fw_cap[0]; + + cap_id = iris_get_cap_id(ctrl->id); + if (!iris_valid_cap_id(cap_id)) + return -EINVAL; + + cap[cap_id].flags |= CAP_FLAG_CLIENT_SET; + + inst->fw_cap[cap_id].value = ctrl->val; + + return 0; +} + +static const struct v4l2_ctrl_ops iris_ctrl_ops = { + .s_ctrl = iris_vdec_op_s_ctrl, + .g_volatile_ctrl = iris_vdec_op_g_volatile_ctrl, +}; + +int iris_ctrls_init(struct iris_inst *inst) +{ + struct platform_inst_fw_cap *cap; + int num_ctrls = 0, ctrl_idx = 0; + int idx = 0, ret; + u32 v4l2_id; + + cap = &inst->fw_cap[0]; + + for (idx = 1; idx < INST_FW_CAP_MAX; idx++) { + if (iris_get_v4l2_id(cap[idx].cap_id)) + num_ctrls++; + } + if (!num_ctrls) + return -EINVAL; + + ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, num_ctrls); + if (ret) + return ret; + + for (idx = 1; idx < INST_FW_CAP_MAX; idx++) { + struct v4l2_ctrl *ctrl; + + v4l2_id = iris_get_v4l2_id(cap[idx].cap_id); + if (!v4l2_id) + continue; + + if (ctrl_idx >= num_ctrls) { + ret = -EINVAL; + goto error; + } + + if (cap[idx].flags & CAP_FLAG_MENU) { + ctrl = v4l2_ctrl_new_std_menu(&inst->ctrl_handler, + &iris_ctrl_ops, + v4l2_id, + cap[idx].max, + ~(cap[idx].step_or_mask), + cap[idx].value); + } else { + ctrl = v4l2_ctrl_new_std(&inst->ctrl_handler, + &iris_ctrl_ops, + v4l2_id, + cap[idx].min, + cap[idx].max, + cap[idx].step_or_mask, + cap[idx].value); + } + if (!ctrl) { + ret = -EINVAL; + goto error; + } + + ret = inst->ctrl_handler.error; + if (ret) + goto error; + + if ((cap[idx].flags & CAP_FLAG_VOLATILE) || + (ctrl->id == V4L2_CID_MIN_BUFFERS_FOR_CAPTURE || + ctrl->id == V4L2_CID_MIN_BUFFERS_FOR_OUTPUT)) + ctrl->flags |= V4L2_CTRL_FLAG_VOLATILE; + + ctrl->flags |= V4L2_CTRL_FLAG_EXECUTE_ON_WRITE; + ctrl_idx++; + } + + return 0; +error: + v4l2_ctrl_handler_free(&inst->ctrl_handler); + + return ret; +} + +int iris_session_init_caps(struct iris_core *core) +{ + struct platform_inst_fw_cap *inst_plat_cap_data; + int i, num_inst_cap; + u32 cap_id; + + inst_plat_cap_data = core->iris_platform_data->inst_fw_cap_data; + if (!inst_plat_cap_data) + return -EINVAL; + + num_inst_cap = core->iris_platform_data->inst_fw_cap_data_size; + + for (i = 0; i < num_inst_cap && i < INST_FW_CAP_MAX - 1; i++) { + cap_id = inst_plat_cap_data[i].cap_id; + if (!iris_valid_cap_id(cap_id)) + continue; + + core->inst_fw_cap[cap_id].cap_id = inst_plat_cap_data[i].cap_id; + core->inst_fw_cap[cap_id].min = inst_plat_cap_data[i].min; + core->inst_fw_cap[cap_id].max = inst_plat_cap_data[i].max; + core->inst_fw_cap[cap_id].step_or_mask = inst_plat_cap_data[i].step_or_mask; + core->inst_fw_cap[cap_id].value = inst_plat_cap_data[i].value; + core->inst_fw_cap[cap_id].flags = inst_plat_cap_data[i].flags; + core->inst_fw_cap[cap_id].hfi_id = inst_plat_cap_data[i].hfi_id; + } + + return 0; +} diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.h b/drivers/media/platform/qcom/iris/iris_ctrls.h new file mode 100644 index 000000000000..46e1da847aa8 --- /dev/null +++ b/drivers/media/platform/qcom/iris/iris_ctrls.h @@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef _IRIS_CTRLS_H_ +#define _IRIS_CTRLS_H_ + +struct iris_core; +struct iris_inst; + +int iris_ctrls_init(struct iris_inst *inst); +int iris_session_init_caps(struct iris_core *core); + +#endif diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h index da52e497b74a..9dc050063924 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h @@ -31,9 +31,13 @@ #define HFI_EVENT_SYS_ERROR 0x1 #define HFI_EVENT_SESSION_ERROR 0x2 +#define HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER 0x1200001 + #define HFI_PROPERTY_SYS_CODEC_POWER_PLANE_CTRL 0x5 #define HFI_PROPERTY_SYS_IMAGE_VERSION 0x6 +#define HFI_PROPERTY_PARAM_WORK_MODE 0x1015 +#define HFI_PROPERTY_PARAM_WORK_ROUTE 0x1017 #define HFI_MSG_SYS_INIT 0x20001 #define HFI_MSG_SYS_SESSION_INIT 0x20006 #define HFI_MSG_SYS_SESSION_END 0x20007 diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c index a74114b0761a..6ad2ca7be0f0 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c @@ -108,6 +108,7 @@ static int iris_hfi_gen2_session_set_default_header(struct iris_inst *inst) struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); u32 default_header = false; + default_header = inst->fw_cap[DEFAULT_HEADER].value; iris_hfi_gen2_packet_session_property(inst, HFI_PROP_DEC_DEFAULT_HEADER, HFI_HOST_FLAGS_NONE, diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h index 18cc9365ab75..401df7b4e976 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h @@ -28,7 +28,16 @@ #define HFI_PROP_UBWC_BANK_SWZL_LEVEL3 0x03000008 #define HFI_PROP_UBWC_BANK_SPREADING 0x03000009 #define HFI_PROP_CODEC 0x03000100 +#define HFI_PROP_PROFILE 0x03000107 +#define HFI_PROP_LEVEL 0x03000108 +#define HFI_PROP_STAGE 0x0300010a +#define HFI_PROP_PIPE 0x0300010b +#define HFI_PROP_LUMA_CHROMA_BIT_DEPTH 0x0300010f +#define HFI_PROP_CODED_FRAMES 0x03000120 +#define HFI_PROP_BUFFER_HOST_MAX_COUNT 0x03000123 +#define HFI_PROP_PIC_ORDER_CNT_TYPE 0x03000128 #define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168 +#define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169 #define HFI_PROP_END 0x03FFFFFF #define HFI_SESSION_ERROR_BEGIN 0x04000000 diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h index d28b8fd7ec2f..2429b9860789 100644 --- a/drivers/media/platform/qcom/iris/iris_instance.h +++ b/drivers/media/platform/qcom/iris/iris_instance.h @@ -23,8 +23,11 @@ * @fh: reference of v4l2 file handler * @fmt_src: structure of v4l2_format for source * @fmt_dst: structure of v4l2_format for destination + * @ctrl_handler: reference of v4l2 ctrl handler * @crop: structure of crop info * @completions: structure of signal completions + * @driver_cap: array of supported instance driver capabilities + * @fw_cap: array of supported instance firmware capabilities * @buffers: array of different iris buffers * @fw_min_count: minimnum count of buffers needed by fw * @once_per_session_set: boolean to set once per session property @@ -42,8 +45,11 @@ struct iris_inst { struct v4l2_fh fh; struct v4l2_format *fmt_src; struct v4l2_format *fmt_dst; + struct v4l2_ctrl_handler ctrl_handler; struct iris_hfi_rect_desc crop; struct completion completion; + struct platform_inst_driver_cap driver_cap[INST_DRIVER_CAP_MAX]; + struct platform_inst_fw_cap fw_cap[INST_FW_CAP_MAX]; struct iris_buffers buffers[BUF_TYPE_MAX]; u32 fw_min_count; bool once_per_session_set; diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h index 754cccc638a5..2935b769abb7 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_common.h +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h @@ -10,6 +10,23 @@ #define HW_RESPONSE_TIMEOUT_VALUE (1000) /* milliseconds */ #define AUTOSUSPEND_DELAY_VALUE (HW_RESPONSE_TIMEOUT_VALUE + 500) /* milliseconds */ +#define REGISTER_BIT_DEPTH(luma, chroma) ((luma) << 16 | (chroma)) +#define BIT_DEPTH_8 REGISTER_BIT_DEPTH(8, 8) +#define CODED_FRAMES_PROGRESSIVE 0x0 +#define DEFAULT_MAX_HOST_BUF_COUNT 64 +#define DEFAULT_MAX_HOST_BURST_BUF_COUNT 256 + +enum stage_type { + STAGE_1 = 1, + STAGE_2 = 2, +}; + +enum pipe_type { + PIPE_1 = 1, + PIPE_2 = 2, + PIPE_4 = 4, +}; + extern struct iris_platform_data sm8550_data; extern struct iris_platform_data sm8250_data; @@ -41,6 +58,56 @@ struct ubwc_config_data { u32 bank_spreading; }; +enum platform_inst_driver_cap_type { + FRAME_WIDTH = 1, + FRAME_HEIGHT, + MBPF, + INST_DRIVER_CAP_MAX, +}; + +enum platform_inst_fw_cap_type { + PROFILE = 1, + LEVEL, + INPUT_BUF_HOST_MAX_COUNT, + STAGE, + PIPE, + POC, + CODED_FRAMES, + BIT_DEPTH, + DEFAULT_HEADER, + RAP_FRAME, + DEBLOCK, + INST_FW_CAP_MAX, +}; + +enum platform_inst_cap_flags { + CAP_FLAG_NONE = 0, + CAP_FLAG_DYNAMIC_ALLOWED = BIT(0), + CAP_FLAG_MENU = BIT(1), + CAP_FLAG_INPUT_PORT = BIT(2), + CAP_FLAG_OUTPUT_PORT = BIT(3), + CAP_FLAG_CLIENT_SET = BIT(4), + CAP_FLAG_BITMASK = BIT(5), + CAP_FLAG_VOLATILE = BIT(6), +}; + +struct platform_inst_driver_cap { + enum platform_inst_driver_cap_type cap_id; + u32 min; + u32 max; + u32 value; +}; + +struct platform_inst_fw_cap { + enum platform_inst_fw_cap_type cap_id; + s64 min; + s64 max; + s64 step_or_mask; + s64 value; + u32 hfi_id; + enum platform_inst_cap_flags flags; +}; + struct iris_core_power { u64 clk_freq; u64 icc_bw; @@ -70,6 +137,10 @@ struct iris_platform_data { u64 dma_mask; const char *fwname; u32 pas_id; + struct platform_inst_driver_cap *inst_driver_cap_data; + u32 inst_driver_cap_data_size; + struct platform_inst_fw_cap *inst_fw_cap_data; + u32 inst_fw_cap_data_size; struct tz_cp_config *tz_cp_config_data; u32 core_arch; u32 hw_response_timeout; diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c index cbc5e84641f6..ac9f720aff8e 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c @@ -7,8 +7,60 @@ #include "iris_platform_common.h" #include "iris_resources.h" #include "iris_hfi_gen1.h" +#include "iris_hfi_gen1_defines.h" #include "iris_vpu_common.h" +static struct platform_inst_driver_cap instance_driver_cap_data_sm8250[] = { + { + .cap_id = FRAME_WIDTH, + .min = 128, + .max = 8192, + .value = 1920, + }, + { + .cap_id = FRAME_HEIGHT, + .min = 128, + .max = 8192, + .value = 1920, + }, + { + .cap_id = MBPF, + .min = 64, + .max = 138240, + .value = 138240, + }, +}; + +static struct platform_inst_fw_cap instance_fw_cap_data_sm8250[] = { + { + .cap_id = PIPE, + .min = PIPE_1, + .max = PIPE_4, + .step_or_mask = 1, + .value = PIPE_4, + .hfi_id = HFI_PROPERTY_PARAM_WORK_ROUTE, + .flags = CAP_FLAG_NONE, + }, + { + .cap_id = STAGE, + .min = STAGE_1, + .max = STAGE_2, + .step_or_mask = 1, + .value = STAGE_2, + .hfi_id = HFI_PROPERTY_PARAM_WORK_MODE, + .flags = CAP_FLAG_NONE, + }, + { + .cap_id = DEBLOCK, + .min = 0, + .max = 1, + .step_or_mask = 1, + .value = 0, + .hfi_id = HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER, + .flags = CAP_FLAG_NONE, + }, +}; + static void iris_set_sm8250_preset_registers(struct iris_core *core) { writel(0x0, core->reg_base + 0xB0088); @@ -57,6 +109,10 @@ struct iris_platform_data sm8250_data = { .dma_mask = GENMASK(31, 29) - 1, .fwname = "qcom/vpu/vpu20_p4.mbn", .pas_id = IRIS_PAS_ID, + .inst_driver_cap_data = instance_driver_cap_data_sm8250, + .inst_driver_cap_data_size = ARRAY_SIZE(instance_driver_cap_data_sm8250), + .inst_fw_cap_data = instance_fw_cap_data_sm8250, + .inst_fw_cap_data_size = ARRAY_SIZE(instance_fw_cap_data_sm8250), .tz_cp_config_data = &tz_cp_config_sm8250, .hw_response_timeout = HW_RESPONSE_TIMEOUT_VALUE, .num_vpp_pipe = 4, diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c index 57fe9986b8cf..42e44b454896 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c @@ -5,12 +5,146 @@ #include "iris_core.h" #include "iris_hfi_gen2.h" +#include "iris_hfi_gen2_defines.h" #include "iris_platform_common.h" #include "iris_resources.h" #include "iris_vpu_common.h" #define VIDEO_ARCH_LX 1 +static struct platform_inst_driver_cap instance_driver_cap_data_sm8550[] = { + { + .cap_id = FRAME_WIDTH, + .min = 96, + .max = 8192, + .value = 1920, + }, + { + .cap_id = FRAME_HEIGHT, + .min = 96, + .max = 8192, + .value = 1920, + }, + { + .cap_id = MBPF, + .min = 36, + .max = 138240, + .value = 138240, + }, +}; + +static struct platform_inst_fw_cap instance_fw_cap_data_sm8550[] = { + { + .cap_id = PROFILE, + .min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, + .max = V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH, + .step_or_mask = BIT(V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE) | + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_HIGH) | + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE) | + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_MAIN) | + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_HIGH), + .value = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + .hfi_id = HFI_PROP_PROFILE, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + }, + { + .cap_id = LEVEL, + .min = V4L2_MPEG_VIDEO_H264_LEVEL_1_0, + .max = V4L2_MPEG_VIDEO_H264_LEVEL_6_2, + .step_or_mask = BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1B) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_1_3) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_2_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_3_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_4_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_5_2) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_0) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_1) | + BIT(V4L2_MPEG_VIDEO_H264_LEVEL_6_2), + .value = V4L2_MPEG_VIDEO_H264_LEVEL_6_1, + .hfi_id = HFI_PROP_LEVEL, + .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + }, + { + .cap_id = INPUT_BUF_HOST_MAX_COUNT, + .min = DEFAULT_MAX_HOST_BUF_COUNT, + .max = DEFAULT_MAX_HOST_BURST_BUF_COUNT, + .step_or_mask = 1, + .value = DEFAULT_MAX_HOST_BUF_COUNT, + .hfi_id = HFI_PROP_BUFFER_HOST_MAX_COUNT, + .flags = CAP_FLAG_INPUT_PORT, + }, + { + .cap_id = STAGE, + .min = STAGE_1, + .max = STAGE_2, + .step_or_mask = 1, + .value = STAGE_2, + .hfi_id = HFI_PROP_STAGE, + .flags = CAP_FLAG_NONE, + }, + { + .cap_id = PIPE, + .min = PIPE_1, + .max = PIPE_4, + .step_or_mask = 1, + .value = PIPE_4, + .hfi_id = HFI_PROP_PIPE, + .flags = CAP_FLAG_NONE, + }, + { + .cap_id = POC, + .min = 0, + .max = 2, + .step_or_mask = 1, + .value = 1, + .hfi_id = HFI_PROP_PIC_ORDER_CNT_TYPE, + }, + { + .cap_id = CODED_FRAMES, + .min = CODED_FRAMES_PROGRESSIVE, + .max = CODED_FRAMES_PROGRESSIVE, + .step_or_mask = 0, + .value = CODED_FRAMES_PROGRESSIVE, + .hfi_id = HFI_PROP_CODED_FRAMES, + }, + { + .cap_id = BIT_DEPTH, + .min = BIT_DEPTH_8, + .max = BIT_DEPTH_8, + .step_or_mask = 1, + .value = BIT_DEPTH_8, + .hfi_id = HFI_PROP_LUMA_CHROMA_BIT_DEPTH, + }, + { + .cap_id = DEFAULT_HEADER, + .min = 0, + .max = 1, + .step_or_mask = 1, + .value = 0, + .hfi_id = HFI_PROP_DEC_DEFAULT_HEADER, + }, + { + .cap_id = RAP_FRAME, + .min = 0, + .max = 1, + .step_or_mask = 1, + .value = 1, + .hfi_id = HFI_PROP_DEC_START_FROM_RAP_FRAME, + .flags = CAP_FLAG_INPUT_PORT, + }, +}; + static void iris_set_sm8550_preset_registers(struct iris_core *core) { writel(0x0, core->reg_base + 0xB0088); @@ -69,6 +203,10 @@ struct iris_platform_data sm8550_data = { .dma_mask = GENMASK(31, 29) - 1, .fwname = "qcom/vpu/vpu30_p4.mbn", .pas_id = IRIS_PAS_ID, + .inst_driver_cap_data = instance_driver_cap_data_sm8550, + .inst_driver_cap_data_size = ARRAY_SIZE(instance_driver_cap_data_sm8550), + .inst_fw_cap_data = instance_fw_cap_data_sm8550, + .inst_fw_cap_data_size = ARRAY_SIZE(instance_fw_cap_data_sm8550), .tz_cp_config_data = &tz_cp_config_sm8550, .core_arch = VIDEO_ARCH_LX, .hw_response_timeout = HW_RESPONSE_TIMEOUT_VALUE, diff --git a/drivers/media/platform/qcom/iris/iris_probe.c b/drivers/media/platform/qcom/iris/iris_probe.c index 5d492b3919cc..319851c50ccd 100644 --- a/drivers/media/platform/qcom/iris/iris_probe.c +++ b/drivers/media/platform/qcom/iris/iris_probe.c @@ -7,6 +7,7 @@ #include #include "iris_core.h" +#include "iris_ctrls.h" #include "iris_vidc.h" static inline int iris_init_isr(struct iris_core *core) @@ -145,6 +146,12 @@ static int iris_probe(struct platform_device *pdev) goto err_runtime_disable; } + ret = iris_session_init_caps(core); + if (ret) { + dev_err_probe(core->dev, ret, "init inst caps failed\n"); + goto err_runtime_disable; + } + ret = v4l2_device_register(dev, &core->v4l2_dev); if (ret) goto err_runtime_disable; diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index 29c7e229c148..f82a337b8f43 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -7,6 +7,7 @@ #include #include "iris_buffer.h" +#include "iris_ctrls.h" #include "iris_instance.h" #include "iris_vdec.h" #include "iris_vpu_buffer.h" @@ -16,9 +17,13 @@ #define DEFAULT_CODEC_ALIGNMENT 16 #define MAX_EVENTS 30 -void iris_vdec_inst_init(struct iris_inst *inst) +int iris_vdec_inst_init(struct iris_inst *inst) { + struct platform_inst_driver_cap *inst_plat_cap_data; + struct iris_core *core = inst->core; struct v4l2_format *f; + int i, num_inst_cap; + u32 cap_id; inst->fmt_src = kzalloc(sizeof(*inst->fmt_src), GFP_KERNEL); inst->fmt_dst = kzalloc(sizeof(*inst->fmt_dst), GFP_KERNEL); @@ -54,6 +59,23 @@ void iris_vdec_inst_init(struct iris_inst *inst) inst->buffers[BUF_OUTPUT].min_count = iris_vpu_buf_count(inst, BUF_OUTPUT); inst->buffers[BUF_OUTPUT].actual_count = inst->buffers[BUF_OUTPUT].min_count; inst->buffers[BUF_OUTPUT].size = f->fmt.pix_mp.plane_fmt[0].sizeimage; + + memcpy(&inst->fw_cap[0], &core->inst_fw_cap[0], + INST_FW_CAP_MAX * sizeof(struct platform_inst_fw_cap)); + + inst_plat_cap_data = core->iris_platform_data->inst_driver_cap_data; + num_inst_cap = core->iris_platform_data->inst_driver_cap_data_size; + + for (i = 0; i < num_inst_cap && i < INST_DRIVER_CAP_MAX - 1; i++) { + cap_id = inst_plat_cap_data[i].cap_id; + + inst->driver_cap[cap_id].cap_id = inst_plat_cap_data[i].cap_id; + inst->driver_cap[cap_id].min = inst_plat_cap_data[i].min; + inst->driver_cap[cap_id].max = inst_plat_cap_data[i].max; + inst->driver_cap[cap_id].value = inst_plat_cap_data[i].value; + } + + return iris_ctrls_init(inst); } void iris_vdec_inst_deinit(struct iris_inst *inst) diff --git a/drivers/media/platform/qcom/iris/iris_vdec.h b/drivers/media/platform/qcom/iris/iris_vdec.h index 707fff34bf4d..d7b8a0ad6fa8 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.h +++ b/drivers/media/platform/qcom/iris/iris_vdec.h @@ -8,7 +8,7 @@ struct iris_inst; -void iris_vdec_inst_init(struct iris_inst *inst); +int iris_vdec_inst_init(struct iris_inst *inst); void iris_vdec_inst_deinit(struct iris_inst *inst); int iris_vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f); int iris_vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f); diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index 141704512b3c..7d5da30cb1d1 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -16,10 +16,6 @@ #define IRIS_DRV_NAME "iris_driver" #define IRIS_BUS_NAME "platform:iris_icc" -#define MIN_FRAME_WIDTH 128 -#define MAX_FRAME_WIDTH 8192 -#define MIN_FRAME_HEIGHT 128 -#define MAX_FRAME_HEIGHT 8192 #define STEP_WIDTH 1 #define STEP_HEIGHT 1 @@ -31,6 +27,7 @@ static int iris_v4l2_fh_init(struct iris_inst *inst) return -EINVAL; v4l2_fh_init(&inst->fh, core->vdev_dec); + inst->fh.ctrl_handler = &inst->ctrl_handler; v4l2_fh_add(&inst->fh); return 0; @@ -188,7 +185,7 @@ int iris_open(struct file *filp) goto fail_m2m_release; } - iris_vdec_inst_init(inst); + ret = iris_vdec_inst_init(inst); if (ret) goto fail_m2m_ctx_release; @@ -244,6 +241,7 @@ int iris_close(struct file *filp) if (!inst) return -EINVAL; + v4l2_ctrl_handler_free(&inst->ctrl_handler); v4l2_m2m_ctx_release(inst->m2m_ctx); v4l2_m2m_release(inst->m2m_dev); mutex_lock(&inst->lock); @@ -351,11 +349,11 @@ static int iris_enum_framesizes(struct file *filp, void *fh, } fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE; - fsize->stepwise.min_width = MIN_FRAME_WIDTH; - fsize->stepwise.max_width = MAX_FRAME_WIDTH; + fsize->stepwise.min_width = inst->driver_cap[FRAME_WIDTH].min; + fsize->stepwise.max_width = inst->driver_cap[FRAME_WIDTH].max; fsize->stepwise.step_width = STEP_WIDTH; - fsize->stepwise.min_height = MIN_FRAME_HEIGHT; - fsize->stepwise.max_height = MAX_FRAME_HEIGHT; + fsize->stepwise.min_height = inst->driver_cap[FRAME_HEIGHT].min; + fsize->stepwise.max_height = inst->driver_cap[FRAME_HEIGHT].max; fsize->stepwise.step_height = STEP_HEIGHT; unlock: From patchwork Tue Aug 27 10:05:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 824188 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F00291A00FE; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; cv=none; b=dM/2JnH0Np2xC7jnW1iWWTyWVok/jCwbubzegSjwhjujYgvDwTACWQibzYsmtmp8PBRODM9Eqmcrrv9wOWNPWOVh/0jdvp1kQEL9EMOvLLXPCAL3RdClXtBxmibVC5FO2iMeHDS4/ULRUcLW4MXHQ6fDi6AP7FC8T48sdnR2bXo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; c=relaxed/simple; bh=7dmUrVm5MsRpRlIfq+Rg5Fs3RAMtLYqQAfzZ0ymvEvo=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=bkRzY+SpcWt/2TZ4kRnrjfiUPliERwXh6NDZzLrwwcl1wQZFxrc612Sw7M7nMxSE4b1d4r0jL2qXCMjscyOTA1UV3cWFmRwFbtZv2psaoGYKG3EetNTr9T3pehKeRxRPuhFOEWDtyyv1Z8vqx6oY6lYzZn6hccP0Ne+L4uL8LeE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hlQcVIVt; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hlQcVIVt" Received: by smtp.kernel.org (Postfix) with ESMTPS id BF912C4FDF3; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753195; bh=7dmUrVm5MsRpRlIfq+Rg5Fs3RAMtLYqQAfzZ0ymvEvo=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=hlQcVIVtx+CsfV5ZPYh274vC2RFGDFEecxTtyWvPCsiwDXp1cONSpLtGEE7vKptjA uqMYiqSyLfCyZ7j/qo3+eV5LMuY7I43fkR4V8F6gSPx9W3EWRbICLGVbmJs01pSpu8 iwz+1ve3zuh53k2Y8/MN1/d/aj5ZTV6sjRd/aGsge9JTx6MWe+xE9aLDBk9NBTXZOk xsEG4VRqXpu6LYnJPTKKNQRqZBlEvL80GEz6aZAEwVb0zZsIeF40hJ+OgmzQYbZUIz 9x5rjXiXr0m6lR3p1lR7eZuCVACNPMg+EUhq7D30iwmcALRRpQFU4ObTpmXApuauJa WQVNO4mPlLmzA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B52EFC54735; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:42 +0530 Subject: [PATCH v3 17/29] media: iris: implement query_cap, query_ctrl and query_menu ioctls Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-17-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal , Vedang Nagar X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=3331; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=NYDzrot9plHMXRI9hRDmFqnK2WCrDqJ1PWcxNJaBut8=; b=bqp6YfioGmZgpf/dd6qIzyzqTSUxRkhxQCLhw+E/xHqnmBGupWizoKvcCw6xbrjsSS+EkCDm0 IeRGLAZLuYsD0qwuG1Ki2Z/HzhrYRxD/KSIE+u8NQIFHvtJKTE4vp7u X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Vedang Nagar Implement query_cap, query_ctrl and query_menu ioctls in the driver with necessary hooks. Signed-off-by: Vedang Nagar Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/iris_vidc.c | 89 ++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index 7d5da30cb1d1..1dd612b7cec5 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -362,6 +362,92 @@ static int iris_enum_framesizes(struct file *filp, void *fh, return ret; } +static int iris_querycap(struct file *filp, void *fh, struct v4l2_capability *cap) +{ + struct iris_inst *inst; + int ret = 0; + + inst = iris_get_inst(filp, fh); + if (!inst) + return -EINVAL; + + mutex_lock(&inst->lock); + strscpy(cap->driver, IRIS_DRV_NAME, sizeof(cap->driver)); + strscpy(cap->bus_info, IRIS_BUS_NAME, sizeof(cap->bus_info)); + memset(cap->reserved, 0, sizeof(cap->reserved)); + strscpy(cap->card, "iris_decoder", sizeof(cap->card)); + mutex_unlock(&inst->lock); + + return ret; +} + +static int iris_queryctrl(struct file *filp, void *fh, struct v4l2_queryctrl *q_ctrl) +{ + struct v4l2_ctrl *ctrl; + struct iris_inst *inst; + int ret = 0; + + inst = iris_get_inst(filp, fh); + if (!inst || !q_ctrl) + return -EINVAL; + + mutex_lock(&inst->lock); + ctrl = v4l2_ctrl_find(&inst->ctrl_handler, q_ctrl->id); + if (!ctrl) { + ret = -EINVAL; + goto unlock; + } + + q_ctrl->minimum = ctrl->minimum; + q_ctrl->maximum = ctrl->maximum; + q_ctrl->default_value = ctrl->default_value; + q_ctrl->flags = 0; + q_ctrl->step = ctrl->step; + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + +static int iris_querymenu(struct file *filp, void *fh, struct v4l2_querymenu *qmenu) +{ + struct v4l2_ctrl *ctrl; + struct iris_inst *inst; + int ret = 0; + + inst = iris_get_inst(filp, fh); + if (!inst || !qmenu) + return -EINVAL; + + mutex_lock(&inst->lock); + ctrl = v4l2_ctrl_find(&inst->ctrl_handler, qmenu->id); + if (!ctrl) { + ret = -EINVAL; + goto unlock; + } + + if (ctrl->type != V4L2_CTRL_TYPE_MENU) { + ret = -EINVAL; + goto unlock; + } + + if (qmenu->index < ctrl->minimum || qmenu->index > ctrl->maximum) { + ret = -EINVAL; + goto unlock; + } + + if (ctrl->menu_skip_mask & (1 << qmenu->index)) { + ret = -EINVAL; + goto unlock; + } + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + static int iris_g_selection(struct file *filp, void *fh, struct v4l2_selection *s) { struct iris_inst *inst; @@ -453,6 +539,9 @@ static const struct v4l2_ioctl_ops iris_v4l2_ioctl_ops = { .vidioc_g_fmt_vid_out_mplane = iris_g_fmt_vid_mplane, .vidioc_enum_framesizes = iris_enum_framesizes, .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs, + .vidioc_querycap = iris_querycap, + .vidioc_queryctrl = iris_queryctrl, + .vidioc_querymenu = iris_querymenu, .vidioc_g_selection = iris_g_selection, .vidioc_subscribe_event = iris_subscribe_event, .vidioc_unsubscribe_event = iris_unsubscribe_event, From patchwork Tue Aug 27 10:05:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 822722 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 152E71A01C4; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; cv=none; b=L9Frg8rwIJQgak6PvnGoZIAUqzKQij/qYqGDPlcVMv7Br6VSXtdUMbb345pK4so/IYIuKbuboIKmu+VlMvl6omK061Sjb3wcmwrIc05F9zluH1FoLBxLZXPxaIgLsQBM5WBbm7WDkt/yfk2Q/yNO5qFJcdQ9Sj4Juh02hYLW6jA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; c=relaxed/simple; bh=aqUoQclyvIXJU156YelZ2I48Pj7nYby6yN3DdJV51+0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=sMqFYj29ZdzuhU6BZf1BUN2vi/i4vTlj/REPoxXBpq3wcWNB9c/hNaulpgeHJdWvOJtSmScmBtXz/0de5yEkwoAFJJ258nZVyQn+ROf0EGsdo3ZH/KNO6ApMydvzO6SOxgyYqdtKzmlV8IxgH7GVBLC3vFyoFuv0Z6Y6SGPU3R0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=M2r0gcQ4; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="M2r0gcQ4" Received: by smtp.kernel.org (Postfix) with ESMTPS id D84A9C4FDF8; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753195; bh=aqUoQclyvIXJU156YelZ2I48Pj7nYby6yN3DdJV51+0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=M2r0gcQ4rplAwG3u3SgTK4lPZQ6QmnqB3nmG/FewXdEisonppu1YN8wbDrnde7zP3 fgNx6riT9W2W3wFMq2c7aR4br5n9PptLR9wV61VbB0QUEk5PZ6bp9ZLcJH63iTQfkj ym9mcT2zO8TPbG1IWLWnUIryIQRTCI5d1Gz3CgA4r9gSpGUVeLsozTv4mcxE8LpM0c NP+fyOkCWCJDCIEhxWs414VH7gvWIuVIoyxUuF678f+vAok0KsMjt589px4NM1bFKG b2FdEuUb2Txqg5sq1MG4mQ0PJduPeeUMwMB7/wf0B8hcNq+20MLRYgjAPKpz4eC35t NPrSqhdrpUfyg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id CFAABC5472E; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:43 +0530 Subject: [PATCH v3 18/29] media: iris: implement vb2 streaming ops Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-18-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=40628; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=3iYYextigobuDNu8RtJpo/cs+TqbFw3+By4eXwG8awE=; b=kTgdFcxjHR9VI+B8tbnTY84mkLthMqU9TNMmjTshRx6a0KD69UDC7yDobxWDWnDaVy3aeIMRH +cWo5ct+Jr/CTrhaa+1jkyqdkVzw+InedNta/4K3v6j/BzkJVzR8h5O X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal In stream on, send HFI_CMD_START on capture and output planes to start the processing on respective planes. During stream off, send HFI_CMD_STOP to firmware which is a synchronous command. After the response is received from firmware, the session is closed on firmware. Introduce different states for instance and state transitions. IRIS_INST_INIT - video instance is opened. IRIS_INST_INPUT_STREAMING - stream on is completed on output plane. IRIS_INST_OUTPUT_STREAMING - stream on is completed on capture plane. IRIS_INST_STREAMING - stream on is completed on both output and capture planes. IRIS_INST_DEINIT - video instance is closed. IRIS_INST_ERROR - error state. | v ------------- +---------| INIT |--------- + | ------------- | | ^ ^ | | / \ | | / \ | | v v | | ----------- ----------- | | | INPUT OUTPUT | | |---| STREAMING STREAMING |---| | ----------- ----------- | | ^ ^ | | \ / | | \ / | | v v | | ------------- | |--------| STREAMING |-----------| | ------------- | | | | | | | | v | | ----------- | +-------->| DEINIT |<----------+ | ----------- | | | | | | | | v | | ---------- | +-------->| ERROR |<-----------+ ----------. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/iris_hfi_common.h | 2 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 82 +++++++++++++++- .../platform/qcom/iris/iris_hfi_gen1_defines.h | 26 ++++- .../platform/qcom/iris/iris_hfi_gen1_response.c | 32 ++++++- .../platform/qcom/iris/iris_hfi_gen2_command.c | 67 +++++++++++++ .../platform/qcom/iris/iris_hfi_gen2_defines.h | 2 + .../platform/qcom/iris/iris_hfi_gen2_response.c | 24 ++++- drivers/media/platform/qcom/iris/iris_instance.h | 4 + .../platform/qcom/iris/iris_platform_common.h | 1 + drivers/media/platform/qcom/iris/iris_state.c | 105 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_state.h | 55 +++++++++++ drivers/media/platform/qcom/iris/iris_utils.c | 11 ++- drivers/media/platform/qcom/iris/iris_utils.h | 2 +- drivers/media/platform/qcom/iris/iris_vb2.c | 79 ++++++++++++++++ drivers/media/platform/qcom/iris/iris_vb2.h | 3 + drivers/media/platform/qcom/iris/iris_vdec.c | 75 +++++++++++++++ drivers/media/platform/qcom/iris/iris_vdec.h | 3 + drivers/media/platform/qcom/iris/iris_vidc.c | 80 +++++++++++++++- 18 files changed, 638 insertions(+), 15 deletions(-) diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.h b/drivers/media/platform/qcom/iris/iris_hfi_common.h index f59ce97d5b7e..4ac97692d072 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_common.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_common.h @@ -49,6 +49,8 @@ struct iris_hfi_command_ops { int (*sys_interframe_powercollapse)(struct iris_core *core); int (*sys_pc_prep)(struct iris_core *core); int (*session_open)(struct iris_inst *inst); + int (*session_start)(struct iris_inst *inst, u32 plane); + int (*session_stop)(struct iris_inst *inst, u32 plane); int (*session_close)(struct iris_inst *inst); }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c index f8bd185bb68b..a2e29f03bfee 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c @@ -71,6 +71,9 @@ static int iris_hfi_gen1_session_open(struct iris_inst *inst) struct hfi_session_open_pkt packet; int ret; + if (inst->state != IRIS_INST_DEINIT) + return -EALREADY; + packet.shdr.hdr.size = sizeof(struct hfi_session_open_pkt); packet.shdr.hdr.pkt_type = HFI_CMD_SYS_SESSION_INIT; packet.shdr.session_id = inst->session_id; @@ -83,7 +86,7 @@ static int iris_hfi_gen1_session_open(struct iris_inst *inst) if (ret) return ret; - return iris_wait_for_session_response(inst); + return iris_wait_for_session_response(inst, false); } static void iris_hfi_gen1_packet_session_cmd(struct iris_inst *inst, @@ -104,12 +107,89 @@ static int iris_hfi_gen1_session_close(struct iris_inst *inst) return iris_hfi_queue_cmd_write(inst->core, &packet, packet.shdr.hdr.size); } +static int iris_hfi_gen1_session_start(struct iris_inst *inst, u32 plane) +{ + struct iris_core *core = inst->core; + struct hfi_session_pkt packet; + int ret; + + if (!V4L2_TYPE_IS_OUTPUT(plane)) + return 0; + + reinit_completion(&inst->completion); + iris_hfi_gen1_packet_session_cmd(inst, &packet, HFI_CMD_SESSION_LOAD_RESOURCES); + + ret = iris_hfi_queue_cmd_write(core, &packet, packet.shdr.hdr.size); + if (ret) + return ret; + + ret = iris_wait_for_session_response(inst, false); + if (ret) + return ret; + + reinit_completion(&inst->completion); + iris_hfi_gen1_packet_session_cmd(inst, &packet, HFI_CMD_SESSION_START); + + ret = iris_hfi_queue_cmd_write(core, &packet, packet.shdr.hdr.size); + if (ret) + return ret; + + return iris_wait_for_session_response(inst, false); +} + +static int iris_hfi_gen1_session_stop(struct iris_inst *inst, u32 plane) +{ + struct iris_core *core = inst->core; + struct hfi_session_flush_pkt flush_pkt; + struct hfi_session_pkt pkt; + u32 flush_type; + int ret = 0; + + if ((V4L2_TYPE_IS_OUTPUT(plane) && + inst->state == IRIS_INST_INPUT_STREAMING) || + (V4L2_TYPE_IS_CAPTURE(plane) && + inst->state == IRIS_INST_OUTPUT_STREAMING) || + inst->state == IRIS_INST_ERROR) { + reinit_completion(&inst->completion); + iris_hfi_gen1_packet_session_cmd(inst, &pkt, HFI_CMD_SESSION_STOP); + ret = iris_hfi_queue_cmd_write(core, &pkt, pkt.shdr.hdr.size); + if (!ret) + ret = iris_wait_for_session_response(inst, false); + + reinit_completion(&inst->completion); + iris_hfi_gen1_packet_session_cmd(inst, &pkt, HFI_CMD_SESSION_RELEASE_RESOURCES); + ret = iris_hfi_queue_cmd_write(core, &pkt, pkt.shdr.hdr.size); + if (!ret) + ret = iris_wait_for_session_response(inst, false); + } else if (inst->state == IRIS_INST_STREAMING) { + if (V4L2_TYPE_IS_OUTPUT(plane)) + flush_type = HFI_FLUSH_ALL; + else if (V4L2_TYPE_IS_CAPTURE(plane)) + flush_type = HFI_FLUSH_OUTPUT; + + reinit_completion(&inst->flush_completion); + + flush_pkt.shdr.hdr.size = sizeof(struct hfi_session_flush_pkt); + flush_pkt.shdr.hdr.pkt_type = HFI_CMD_SESSION_FLUSH; + flush_pkt.shdr.session_id = inst->session_id; + flush_pkt.flush_type = flush_type; + + ret = iris_hfi_queue_cmd_write(core, &flush_pkt, flush_pkt.shdr.hdr.size); + if (!ret) + ret = iris_wait_for_session_response(inst, true); + } + + return ret; +} + static const struct iris_hfi_command_ops iris_hfi_gen1_command_ops = { .sys_init = iris_hfi_gen1_sys_init, .sys_image_version = iris_hfi_gen1_sys_image_version, .sys_interframe_powercollapse = iris_hfi_gen1_sys_interframe_powercollapse, .sys_pc_prep = iris_hfi_gen1_sys_pc_prep, .session_open = iris_hfi_gen1_session_open, + .session_start = iris_hfi_gen1_session_start, + .session_stop = iris_hfi_gen1_session_stop, .session_close = iris_hfi_gen1_session_close, }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h index 9dc050063924..fb11d3e45c9f 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h @@ -23,6 +23,12 @@ #define HFI_CMD_SYS_SESSION_INIT 0x10007 #define HFI_CMD_SYS_SESSION_END 0x10008 +#define HFI_CMD_SESSION_LOAD_RESOURCES 0x211001 +#define HFI_CMD_SESSION_START 0x211002 +#define HFI_CMD_SESSION_STOP 0x211003 +#define HFI_CMD_SESSION_FLUSH 0x211008 +#define HFI_CMD_SESSION_RELEASE_RESOURCES 0x21100c + #define HFI_ERR_SESSION_UNSUPPORTED_SETTING 0x1008 #define HFI_ERR_SESSION_UNSUPPORT_BUFFERTYPE 0x1010 #define HFI_ERR_SESSION_INVALID_SCALE_FACTOR 0x1012 @@ -31,8 +37,11 @@ #define HFI_EVENT_SYS_ERROR 0x1 #define HFI_EVENT_SESSION_ERROR 0x2 -#define HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER 0x1200001 +#define HFI_FLUSH_OUTPUT 0x1000002 +#define HFI_FLUSH_OUTPUT2 0x1000003 +#define HFI_FLUSH_ALL 0x1000004 +#define HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER 0x1200001 #define HFI_PROPERTY_SYS_CODEC_POWER_PLANE_CTRL 0x5 #define HFI_PROPERTY_SYS_IMAGE_VERSION 0x6 @@ -45,6 +54,11 @@ #define HFI_MSG_SYS_PROPERTY_INFO 0x2000a #define HFI_MSG_EVENT_NOTIFY 0x21001 +#define HFI_MSG_SESSION_LOAD_RESOURCES 0x221001 +#define HFI_MSG_SESSION_START 0x221002 +#define HFI_MSG_SESSION_STOP 0x221003 +#define HFI_MSG_SESSION_FLUSH 0x221006 +#define HFI_MSG_SESSION_RELEASE_RESOURCES 0x22100a struct hfi_pkt_hdr { u32 size; @@ -87,6 +101,11 @@ struct hfi_sys_pc_prep_pkt { struct hfi_pkt_hdr hdr; }; +struct hfi_session_flush_pkt { + struct hfi_session_hdr_pkt shdr; + u32 flush_type; +}; + struct hfi_msg_event_notify_pkt { struct hfi_session_hdr_pkt shdr; u32 event_id; @@ -120,6 +139,11 @@ struct hfi_msg_sys_property_info_pkt { u8 data[]; }; +struct hfi_msg_session_flush_done_pkt { + struct hfi_msg_session_hdr_pkt shdr; + u32 flush_type; +}; + struct hfi_enable { u32 enable; }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c index c9b87ff4db3a..b42eba10e074 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c @@ -44,6 +44,7 @@ iris_hfi_gen1_event_session_error(struct iris_inst *inst, struct hfi_msg_event_n pkt->event_data2, pkt->event_data1, pkt->shdr.session_id); iris_vb2_queue_error(inst); + iris_inst_change_state(inst, IRIS_INST_ERROR); break; } } @@ -148,6 +149,26 @@ static const struct iris_hfi_gen1_response_pkt_info pkt_infos[] = { .pkt = HFI_MSG_SYS_SESSION_END, .pkt_sz = sizeof(struct hfi_msg_session_hdr_pkt), }, + { + .pkt = HFI_MSG_SESSION_LOAD_RESOURCES, + .pkt_sz = sizeof(struct hfi_msg_session_hdr_pkt), + }, + { + .pkt = HFI_MSG_SESSION_START, + .pkt_sz = sizeof(struct hfi_msg_session_hdr_pkt), + }, + { + .pkt = HFI_MSG_SESSION_STOP, + .pkt_sz = sizeof(struct hfi_msg_session_hdr_pkt), + }, + { + .pkt = HFI_MSG_SESSION_FLUSH, + .pkt_sz = sizeof(struct hfi_msg_session_flush_done_pkt), + }, + { + .pkt = HFI_MSG_SESSION_RELEASE_RESOURCES, + .pkt_sz = sizeof(struct hfi_msg_session_hdr_pkt), + }, }; static void iris_hfi_gen1_handle_response(struct iris_core *core, void *response) @@ -155,6 +176,7 @@ static void iris_hfi_gen1_handle_response(struct iris_core *core, void *response const struct iris_hfi_gen1_response_pkt_info *pkt_info; struct device *dev = core->dev; struct hfi_pkt_hdr *hdr; + struct completion *done; struct iris_inst *inst; bool found = false; unsigned int i; @@ -205,7 +227,15 @@ static void iris_hfi_gen1_handle_response(struct iris_core *core, void *response } mutex_lock(&inst->lock); - complete(&inst->completion); + struct hfi_msg_session_hdr_pkt *shdr; + + shdr = (struct hfi_msg_session_hdr_pkt *)hdr; + if (shdr->error_type != HFI_ERR_NONE) + iris_inst_change_state(inst, IRIS_INST_ERROR); + + done = pkt_info->pkt == HFI_MSG_SESSION_FLUSH ? + &inst->flush_completion : &inst->completion; + complete(done); mutex_unlock(&inst->lock); } } diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c index 6ad2ca7be0f0..5ace377fec7a 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c @@ -85,6 +85,24 @@ static int iris_hfi_gen2_sys_pc_prep(struct iris_core *core) return ret; } +static u32 iris_hfi_gen2_get_port(u32 plane) +{ + u32 hfi_port = HFI_PORT_NONE; + + switch (plane) { + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: + hfi_port = HFI_PORT_BITSTREAM; + break; + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: + hfi_port = HFI_PORT_RAW; + break; + default: + break; + } + + return hfi_port; +} + static int iris_hfi_gen2_session_set_codec(struct iris_inst *inst) { struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); @@ -126,6 +144,9 @@ static int iris_hfi_gen2_session_open(struct iris_inst *inst) struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); int ret; + if (inst->state != IRIS_INST_DEINIT) + return -EALREADY; + inst_hfi_gen2->packet = kzalloc(4096, GFP_KERNEL); if (!inst_hfi_gen2->packet) return -ENOMEM; @@ -190,12 +211,58 @@ static int iris_hfi_gen2_session_close(struct iris_inst *inst) return ret; } +static int iris_hfi_gen2_session_start(struct iris_inst *inst, u32 plane) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + + iris_hfi_gen2_packet_session_command(inst, + HFI_CMD_START, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED), + iris_hfi_gen2_get_port(plane), + inst->session_id, + HFI_PAYLOAD_NONE, + NULL, + 0); + + return iris_hfi_queue_cmd_write(inst->core, inst_hfi_gen2->packet, + inst_hfi_gen2->packet->size); +} + +static int iris_hfi_gen2_session_stop(struct iris_inst *inst, u32 plane) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + int ret = 0; + + reinit_completion(&inst->completion); + + iris_hfi_gen2_packet_session_command(inst, + HFI_CMD_STOP, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED | + HFI_HOST_FLAGS_NON_DISCARDABLE), + iris_hfi_gen2_get_port(plane), + inst->session_id, + HFI_PAYLOAD_NONE, + NULL, + 0); + + ret = iris_hfi_queue_cmd_write(inst->core, inst_hfi_gen2->packet, + inst_hfi_gen2->packet->size); + if (ret) + return ret; + + return iris_wait_for_session_response(inst, false); +} + static const struct iris_hfi_command_ops iris_hfi_gen2_command_ops = { .sys_init = iris_hfi_gen2_sys_init, .sys_image_version = iris_hfi_gen2_sys_image_version, .sys_interframe_powercollapse = iris_hfi_gen2_sys_interframe_powercollapse, .sys_pc_prep = iris_hfi_gen2_sys_pc_prep, .session_open = iris_hfi_gen2_session_open, + .session_start = iris_hfi_gen2_session_start, + .session_stop = iris_hfi_gen2_session_stop, .session_close = iris_hfi_gen2_session_close, }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h index 401df7b4e976..f822b91bc217 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h @@ -15,6 +15,8 @@ #define HFI_CMD_POWER_COLLAPSE 0x01000002 #define HFI_CMD_OPEN 0x01000003 #define HFI_CMD_CLOSE 0x01000004 +#define HFI_CMD_START 0x01000005 +#define HFI_CMD_STOP 0x01000006 #define HFI_CMD_END 0x01FFFFFF #define HFI_PROP_BEGIN 0x03000000 diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c index dce2cf04a856..35ecc66d630a 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c @@ -101,6 +101,7 @@ static int iris_hfi_gen2_handle_session_error(struct iris_inst *inst, dev_err(core->dev, "session error received %#x: %s\n", pkt->type, error); iris_vb2_queue_error(inst); + iris_inst_change_state(inst, IRIS_INST_ERROR); return 0; } @@ -129,20 +130,32 @@ static int iris_hfi_gen2_handle_system_init(struct iris_core *core, return 0; } +static void iris_hfi_gen2_handle_session_close(struct iris_inst *inst, + struct iris_hfi_packet *pkt) +{ + if (!(pkt->flags & HFI_FW_FLAGS_SUCCESS)) { + iris_inst_change_state(inst, IRIS_INST_ERROR); + return; + } + + complete(&inst->completion); +} + static int iris_hfi_gen2_handle_session_command(struct iris_inst *inst, struct iris_hfi_packet *pkt) { - int ret = 0; - switch (pkt->type) { case HFI_CMD_CLOSE: + iris_hfi_gen2_handle_session_close(inst, pkt); + break; + case HFI_CMD_STOP: complete(&inst->completion); break; default: break; } - return ret; + return 0; } static int iris_hfi_gen2_handle_image_version_property(struct iris_core *core, @@ -256,8 +269,11 @@ static int iris_hfi_gen2_handle_session_response(struct iris_core *core, if (packet->flags & HFI_FW_FLAGS_SESSION_ERROR) iris_hfi_gen2_handle_session_error(inst, packet); - if (packet->type > range[i].begin && packet->type < range[i].end) + if (packet->type > range[i].begin && packet->type < range[i].end) { ret = range[i].handle(inst, packet); + if (ret) + iris_inst_change_state(inst, IRIS_INST_ERROR); + } pkt += packet->size; } } diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h index 2429b9860789..c225466b4e48 100644 --- a/drivers/media/platform/qcom/iris/iris_instance.h +++ b/drivers/media/platform/qcom/iris/iris_instance.h @@ -26,10 +26,12 @@ * @ctrl_handler: reference of v4l2 ctrl handler * @crop: structure of crop info * @completions: structure of signal completions + * @flush_completions: structure of signal completions for flush cmd * @driver_cap: array of supported instance driver capabilities * @fw_cap: array of supported instance firmware capabilities * @buffers: array of different iris buffers * @fw_min_count: minimnum count of buffers needed by fw + * @state: instance state * @once_per_session_set: boolean to set once per session property * @m2m_dev: a reference to m2m device structure * @m2m_ctx: a reference to m2m context structure @@ -48,10 +50,12 @@ struct iris_inst { struct v4l2_ctrl_handler ctrl_handler; struct iris_hfi_rect_desc crop; struct completion completion; + struct completion flush_completion; struct platform_inst_driver_cap driver_cap[INST_DRIVER_CAP_MAX]; struct platform_inst_fw_cap fw_cap[INST_FW_CAP_MAX]; struct iris_buffers buffers[BUF_TYPE_MAX]; u32 fw_min_count; + enum iris_inst_state state; bool once_per_session_set; struct v4l2_m2m_dev *m2m_dev; struct v4l2_m2m_ctx *m2m_ctx; diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h index 2935b769abb7..3893e690d314 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_common.h +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h @@ -15,6 +15,7 @@ #define CODED_FRAMES_PROGRESSIVE 0x0 #define DEFAULT_MAX_HOST_BUF_COUNT 64 #define DEFAULT_MAX_HOST_BURST_BUF_COUNT 256 +#define DEFAULT_FPS 30 enum stage_type { STAGE_1 = 1, diff --git a/drivers/media/platform/qcom/iris/iris_state.c b/drivers/media/platform/qcom/iris/iris_state.c index 043aa0ee80b6..14a7f03a012c 100644 --- a/drivers/media/platform/qcom/iris/iris_state.c +++ b/drivers/media/platform/qcom/iris/iris_state.c @@ -3,8 +3,7 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ -#include "iris_core.h" -#include "iris_state.h" +#include "iris_instance.h" void iris_change_core_state(struct iris_core *core, enum iris_core_state request_state) @@ -13,3 +12,105 @@ void iris_change_core_state(struct iris_core *core, core->state = request_state; mutex_unlock(&core->lock); } + +static bool iris_allow_inst_state_change(struct iris_inst *inst, + enum iris_inst_state req_state) +{ + switch (inst->state) { + case IRIS_INST_INIT: + if (req_state == IRIS_INST_INPUT_STREAMING || + req_state == IRIS_INST_OUTPUT_STREAMING || + req_state == IRIS_INST_DEINIT || + req_state == IRIS_INST_ERROR) + return true; + break; + case IRIS_INST_INPUT_STREAMING: + if (req_state == IRIS_INST_INIT || + req_state == IRIS_INST_STREAMING || + req_state == IRIS_INST_DEINIT || + req_state == IRIS_INST_ERROR) + return true; + break; + case IRIS_INST_OUTPUT_STREAMING: + if (req_state == IRIS_INST_INIT || + req_state == IRIS_INST_STREAMING || + req_state == IRIS_INST_DEINIT || + req_state == IRIS_INST_ERROR) + return true; + break; + case IRIS_INST_STREAMING: + if (req_state == IRIS_INST_INPUT_STREAMING || + req_state == IRIS_INST_OUTPUT_STREAMING || + req_state == IRIS_INST_DEINIT || + req_state == IRIS_INST_ERROR) + return true; + break; + case IRIS_INST_DEINIT: + if (req_state == IRIS_INST_INIT || + req_state == IRIS_INST_ERROR) + return true; + break; + default: + return false; + } + + return false; +} + +int iris_inst_change_state(struct iris_inst *inst, + enum iris_inst_state request_state) +{ + if (inst->state == IRIS_INST_ERROR) + return 0; + + if (inst->state == request_state) + return 0; + + if (!iris_allow_inst_state_change(inst, request_state)) + return -EINVAL; + + dev_dbg(inst->core->dev, "state changed from %x to %x\n", + inst->state, request_state); + + inst->state = request_state; + + return 0; +} + +int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane) +{ + enum iris_inst_state new_state = IRIS_INST_ERROR; + + if (V4L2_TYPE_IS_OUTPUT(plane)) { + if (inst->state == IRIS_INST_INIT) + new_state = IRIS_INST_INPUT_STREAMING; + else if (inst->state == IRIS_INST_OUTPUT_STREAMING) + new_state = IRIS_INST_STREAMING; + } else if (V4L2_TYPE_IS_CAPTURE(plane)) { + if (inst->state == IRIS_INST_INIT) + new_state = IRIS_INST_OUTPUT_STREAMING; + else if (inst->state == IRIS_INST_INPUT_STREAMING) + new_state = IRIS_INST_STREAMING; + } + + return iris_inst_change_state(inst, new_state); +} + +int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane) +{ + enum iris_inst_state new_state = IRIS_INST_ERROR; + + if (V4L2_TYPE_IS_OUTPUT(plane)) { + if (inst->state == IRIS_INST_INPUT_STREAMING) + new_state = IRIS_INST_INIT; + else if (inst->state == IRIS_INST_STREAMING) + new_state = IRIS_INST_OUTPUT_STREAMING; + } else if (V4L2_TYPE_IS_CAPTURE(plane)) { + if (inst->state == IRIS_INST_OUTPUT_STREAMING) + new_state = IRIS_INST_INIT; + else if (inst->state == IRIS_INST_STREAMING) + new_state = IRIS_INST_INPUT_STREAMING; + } + + return iris_inst_change_state(inst, new_state); +} diff --git a/drivers/media/platform/qcom/iris/iris_state.h b/drivers/media/platform/qcom/iris/iris_state.h index 8bf22f839311..d6613970a24e 100644 --- a/drivers/media/platform/qcom/iris/iris_state.h +++ b/drivers/media/platform/qcom/iris/iris_state.h @@ -7,6 +7,7 @@ #define _IRIS_STATE_H_ struct iris_core; +struct iris_inst; /** * enum iris_core_state @@ -40,6 +41,60 @@ enum iris_core_state { IRIS_CORE_ERROR, }; +/** + * enum iris_inst_state + * + * IRIS_INST_INIT: video instance is opened. + * IRIS_INST_INPUT_STREAMING: stream on is completed on output plane. + * IRIS_INST_OUTPUT_STREAMING: stream on is completed on capture plane. + * IRIS_INST_STREAMING: stream on is completed on both output and capture planes. + * IRIS_INST_DEINIT: video instance is closed. + * IRIS_INST_ERROR: error state. + * | + * V + * ------------- + * +--------| INIT |----------+ + * | ------------- | + * | ^ ^ | + * | / \ | + * | / \ | + * | v v | + * | ----------- ----------- | + * | | INPUT OUTPUT | | + * |---| STREAMING STREAMING |---| + * | ----------- ----------- | + * | ^ ^ | + * | \ / | + * | \ / | + * | v v | + * | ------------- | + * |--------| STREAMING |-----------| + * | ------------- | + * | | | + * | | | + * | v | + * | ----------- | + * +-------->| DEINIT |<----------+ + * | ----------- | + * | | | + * | | | + * | v | + * | ---------- | + * +-------->| ERROR |<------------+ + * ---------- + */ +enum iris_inst_state { + IRIS_INST_DEINIT, + IRIS_INST_INIT, + IRIS_INST_INPUT_STREAMING, + IRIS_INST_OUTPUT_STREAMING, + IRIS_INST_STREAMING, + IRIS_INST_ERROR, +}; void iris_change_core_state(struct iris_core *core, enum iris_core_state request_state); +int iris_inst_change_state(struct iris_inst *inst, + enum iris_inst_state request_state); +int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane); +int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane); #endif diff --git a/drivers/media/platform/qcom/iris/iris_utils.c b/drivers/media/platform/qcom/iris/iris_utils.c index d7e7c9852aca..47e210ef4beb 100644 --- a/drivers/media/platform/qcom/iris/iris_utils.c +++ b/drivers/media/platform/qcom/iris/iris_utils.c @@ -20,20 +20,23 @@ int iris_get_mbpf(struct iris_inst *inst) return NUM_MBS_PER_FRAME(height, width); } -int iris_wait_for_session_response(struct iris_inst *inst) +int iris_wait_for_session_response(struct iris_inst *inst, bool is_flush) { struct iris_core *core = inst->core; u32 hw_response_timeout_val; + struct completion *done; int ret; hw_response_timeout_val = core->iris_platform_data->hw_response_timeout; + done = is_flush ? &inst->flush_completion : &inst->completion; mutex_unlock(&inst->lock); - ret = wait_for_completion_timeout(&inst->completion, - msecs_to_jiffies(hw_response_timeout_val)); + ret = wait_for_completion_timeout(done, msecs_to_jiffies(hw_response_timeout_val)); mutex_lock(&inst->lock); - if (!ret) + if (!ret) { + iris_inst_change_state(inst, IRIS_INST_ERROR); return -ETIMEDOUT; + } return 0; } diff --git a/drivers/media/platform/qcom/iris/iris_utils.h b/drivers/media/platform/qcom/iris/iris_utils.h index 1c1e109d9b5b..a4a797fb11e2 100644 --- a/drivers/media/platform/qcom/iris/iris_utils.h +++ b/drivers/media/platform/qcom/iris/iris_utils.h @@ -33,6 +33,6 @@ static inline enum iris_buffer_type iris_v4l2_type_to_driver(u32 type) int iris_get_mbpf(struct iris_inst *inst); struct iris_inst *iris_get_instance(struct iris_core *core, u32 session_id); -int iris_wait_for_session_response(struct iris_inst *inst); +int iris_wait_for_session_response(struct iris_inst *inst, bool is_flush); #endif diff --git a/drivers/media/platform/qcom/iris/iris_vb2.c b/drivers/media/platform/qcom/iris/iris_vb2.c index 3fd18b6773fd..03b533aef010 100644 --- a/drivers/media/platform/qcom/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/iris/iris_vb2.c @@ -6,6 +6,7 @@ #include "iris_buffer.h" #include "iris_instance.h" #include "iris_vb2.h" +#include "iris_vdec.h" #include "iris_vpu_buffer.h" int iris_vb2_queue_setup(struct vb2_queue *q, @@ -27,10 +28,19 @@ int iris_vb2_queue_setup(struct vb2_queue *q, return -EINVAL; mutex_lock(&inst->lock); + if (inst->state == IRIS_INST_ERROR) { + ret = -EBUSY; + goto unlock; + } core = inst->core; f = V4L2_TYPE_IS_OUTPUT(q->type) ? inst->fmt_src : inst->fmt_dst; + if (inst->state == IRIS_INST_STREAMING) { + ret = -EINVAL; + goto unlock; + } + if (*num_planes) { if (*num_planes != f->fmt.pix_mp.num_planes || sizes[0] < f->fmt.pix_mp.plane_fmt[0].sizeimage) { @@ -62,6 +72,10 @@ int iris_vb2_queue_setup(struct vb2_queue *q, dev_err(core->dev, "session open failed\n"); goto unlock; } + + ret = iris_inst_change_state(inst, IRIS_INST_INIT); + if (ret) + goto unlock; } buffers = &inst->buffers[buffer_type]; @@ -90,3 +104,68 @@ int iris_vb2_queue_setup(struct vb2_queue *q, return ret; } + +int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count) +{ + struct iris_inst *inst; + int ret = 0; + + if (!q) + return -EINVAL; + + inst = vb2_get_drv_priv(q); + if (!inst || !inst->core) + return -EINVAL; + + mutex_lock(&inst->lock); + if (inst->state == IRIS_INST_ERROR) { + ret = -EBUSY; + goto error; + } + + if (!V4L2_TYPE_IS_OUTPUT(q->type) && + !V4L2_TYPE_IS_CAPTURE(q->type)) { + ret = -EINVAL; + goto error; + } + + if (V4L2_TYPE_IS_OUTPUT(q->type)) + ret = iris_vdec_streamon_input(inst); + else if (V4L2_TYPE_IS_CAPTURE(q->type)) + ret = iris_vdec_streamon_output(inst); + if (ret) + goto error; + + mutex_unlock(&inst->lock); + + return ret; + +error: + iris_inst_change_state(inst, IRIS_INST_ERROR); + mutex_unlock(&inst->lock); + + return ret; +} + +void iris_vb2_stop_streaming(struct vb2_queue *q) +{ + struct iris_inst *inst; + + if (!q) + return; + + inst = vb2_get_drv_priv(q); + if (!inst) + return; + + mutex_lock(&inst->lock); + + if (!V4L2_TYPE_IS_OUTPUT(q->type) && + !V4L2_TYPE_IS_CAPTURE(q->type)) + goto exit; + + iris_vdec_session_streamoff(inst, q->type); + +exit: + mutex_unlock(&inst->lock); +} diff --git a/drivers/media/platform/qcom/iris/iris_vb2.h b/drivers/media/platform/qcom/iris/iris_vb2.h index 78157a97b86e..bc3bb830c2ba 100644 --- a/drivers/media/platform/qcom/iris/iris_vb2.h +++ b/drivers/media/platform/qcom/iris/iris_vb2.h @@ -9,4 +9,7 @@ int iris_vb2_queue_setup(struct vb2_queue *q, unsigned int *num_buffers, unsigned int *num_planes, unsigned int sizes[], struct device *alloc_devs[]); +int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count); +void iris_vb2_stop_streaming(struct vb2_queue *q); + #endif diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index f82a337b8f43..a2c9e9481677 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -255,3 +255,78 @@ int iris_vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_su return ret; } + +static void iris_vdec_kill_session(struct iris_inst *inst) +{ + const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + + if (!inst->session_id) + return; + + hfi_ops->session_close(inst); + iris_inst_change_state(inst, IRIS_INST_ERROR); +} + +void iris_vdec_session_streamoff(struct iris_inst *inst, u32 plane) +{ + const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + int ret; + + ret = hfi_ops->session_stop(inst, plane); + if (ret) + goto error; + + ret = iris_inst_state_change_streamoff(inst, plane); + if (ret) + goto error; + + return; + +error: + iris_vdec_kill_session(inst); +} + +static int iris_vdec_process_streamon_input(struct iris_inst *inst) +{ + const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + int ret; + + ret = hfi_ops->session_start(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + + return iris_inst_state_change_streamon(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); +} + +int iris_vdec_streamon_input(struct iris_inst *inst) +{ + return iris_vdec_process_streamon_input(inst); +} + +static int iris_vdec_process_streamon_output(struct iris_inst *inst) +{ + const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + int ret; + + ret = hfi_ops->session_start(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + if (ret) + return ret; + + return iris_inst_state_change_streamon(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); +} + +int iris_vdec_streamon_output(struct iris_inst *inst) +{ + int ret; + + ret = iris_vdec_process_streamon_output(inst); + if (ret) + goto error; + + return ret; + +error: + iris_vdec_session_streamoff(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + + return ret; +} diff --git a/drivers/media/platform/qcom/iris/iris_vdec.h b/drivers/media/platform/qcom/iris/iris_vdec.h index d7b8a0ad6fa8..b3299164f823 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.h +++ b/drivers/media/platform/qcom/iris/iris_vdec.h @@ -14,5 +14,8 @@ int iris_vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f); int iris_vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f); int iris_vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f); int iris_vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub); +int iris_vdec_streamon_input(struct iris_inst *inst); +int iris_vdec_streamon_output(struct iris_inst *inst); +void iris_vdec_session_streamoff(struct iris_inst *inst, u32 plane); #endif diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index 1dd612b7cec5..2aad96bbaf94 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -164,10 +164,12 @@ int iris_open(struct file *filp) inst->core = core; inst->session_id = hash32_ptr(inst); + inst->state = IRIS_INST_DEINIT; mutex_init(&inst->lock); mutex_init(&inst->ctx_q_lock); init_completion(&inst->completion); + init_completion(&inst->flush_completion); ret = iris_v4l2_fh_init(inst); if (ret) @@ -221,6 +223,9 @@ static void iris_session_close(struct iris_inst *inst) bool wait_for_response; int ret; + if (inst->state == IRIS_INST_DEINIT) + return; + wait_for_response = true; reinit_completion(&inst->completion); @@ -230,7 +235,7 @@ static void iris_session_close(struct iris_inst *inst) wait_for_response = false; if (wait_for_response) - iris_wait_for_session_response(inst); + iris_wait_for_session_response(inst, false); } int iris_close(struct file *filp) @@ -247,6 +252,7 @@ int iris_close(struct file *filp) mutex_lock(&inst->lock); iris_vdec_inst_deinit(inst); iris_session_close(inst); + iris_inst_change_state(inst, IRIS_INST_DEINIT); iris_v4l2_fh_deinit(inst); iris_remove_session(inst); mutex_unlock(&inst->lock); @@ -268,7 +274,14 @@ static int iris_enum_fmt(struct file *filp, void *fh, struct v4l2_fmtdesc *f) return -EINVAL; mutex_lock(&inst->lock); + if (inst->state == IRIS_INST_ERROR) { + ret = -EBUSY; + goto unlock; + } + ret = iris_vdec_enum_fmt(inst, f); + +unlock: mutex_unlock(&inst->lock); return ret; @@ -284,7 +297,14 @@ static int iris_try_fmt_vid_mplane(struct file *filp, void *fh, struct v4l2_form return -EINVAL; mutex_lock(&inst->lock); + if (inst->state == IRIS_INST_ERROR) { + ret = -EBUSY; + goto unlock; + } + ret = iris_vdec_try_fmt(inst, f); + +unlock: mutex_unlock(&inst->lock); return ret; @@ -300,7 +320,14 @@ static int iris_s_fmt_vid_mplane(struct file *filp, void *fh, struct v4l2_format return -EINVAL; mutex_lock(&inst->lock); + if (inst->state == IRIS_INST_ERROR) { + ret = -EBUSY; + goto unlock; + } + ret = iris_vdec_s_fmt(inst, f); + +unlock: mutex_unlock(&inst->lock); return ret; @@ -316,6 +343,11 @@ static int iris_g_fmt_vid_mplane(struct file *filp, void *fh, struct v4l2_format return -EINVAL; mutex_lock(&inst->lock); + if (inst->state == IRIS_INST_ERROR) { + ret = -EBUSY; + goto unlock; + } + if (V4L2_TYPE_IS_OUTPUT(f->type)) memcpy(f, inst->fmt_src, sizeof(*f)); else if (V4L2_TYPE_IS_CAPTURE(f->type)) @@ -323,6 +355,7 @@ static int iris_g_fmt_vid_mplane(struct file *filp, void *fh, struct v4l2_format else ret = -EINVAL; +unlock: mutex_unlock(&inst->lock); return ret; @@ -342,6 +375,11 @@ static int iris_enum_framesizes(struct file *filp, void *fh, return -EINVAL; mutex_lock(&inst->lock); + if (inst->state == IRIS_INST_ERROR) { + ret = -EBUSY; + goto unlock; + } + if (fsize->pixel_format != V4L2_PIX_FMT_H264 && fsize->pixel_format != V4L2_PIX_FMT_NV12) { ret = -EINVAL; @@ -372,10 +410,17 @@ static int iris_querycap(struct file *filp, void *fh, struct v4l2_capability *ca return -EINVAL; mutex_lock(&inst->lock); + if (inst->state == IRIS_INST_ERROR) { + ret = -EBUSY; + goto unlock; + } + strscpy(cap->driver, IRIS_DRV_NAME, sizeof(cap->driver)); strscpy(cap->bus_info, IRIS_BUS_NAME, sizeof(cap->bus_info)); memset(cap->reserved, 0, sizeof(cap->reserved)); strscpy(cap->card, "iris_decoder", sizeof(cap->card)); + +unlock: mutex_unlock(&inst->lock); return ret; @@ -392,6 +437,11 @@ static int iris_queryctrl(struct file *filp, void *fh, struct v4l2_queryctrl *q_ return -EINVAL; mutex_lock(&inst->lock); + if (inst->state == IRIS_INST_ERROR) { + ret = -EBUSY; + goto unlock; + } + ctrl = v4l2_ctrl_find(&inst->ctrl_handler, q_ctrl->id); if (!ctrl) { ret = -EINVAL; @@ -421,6 +471,11 @@ static int iris_querymenu(struct file *filp, void *fh, struct v4l2_querymenu *qm return -EINVAL; mutex_lock(&inst->lock); + if (inst->state == IRIS_INST_ERROR) { + ret = -EBUSY; + goto unlock; + } + ctrl = v4l2_ctrl_find(&inst->ctrl_handler, qmenu->id); if (!ctrl) { ret = -EINVAL; @@ -458,6 +513,11 @@ static int iris_g_selection(struct file *filp, void *fh, struct v4l2_selection * return -EINVAL; mutex_lock(&inst->lock); + if (inst->state == IRIS_INST_ERROR) { + ret = -EBUSY; + goto unlock; + } + if (s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE && s->type != V4L2_BUF_TYPE_VIDEO_CAPTURE) { ret = -EINVAL; @@ -495,7 +555,14 @@ static int iris_subscribe_event(struct v4l2_fh *fh, const struct v4l2_event_subs inst = container_of(fh, struct iris_inst, fh); mutex_lock(&inst->lock); + if (inst->state == IRIS_INST_ERROR) { + ret = -EBUSY; + goto unlock; + } + ret = iris_vdec_subscribe_event(inst, sub); + +unlock: mutex_unlock(&inst->lock); return ret; @@ -509,7 +576,14 @@ static int iris_unsubscribe_event(struct v4l2_fh *fh, const struct v4l2_event_su inst = container_of(fh, struct iris_inst, fh); mutex_lock(&inst->lock); + if (inst->state == IRIS_INST_ERROR) { + ret = -EBUSY; + goto unlock; + } + ret = v4l2_event_unsubscribe(&inst->fh, sub); + +unlock: mutex_unlock(&inst->lock); return ret; @@ -526,6 +600,8 @@ static struct v4l2_file_operations iris_v4l2_file_ops = { static const struct vb2_ops iris_vb2_ops = { .queue_setup = iris_vb2_queue_setup, + .start_streaming = iris_vb2_start_streaming, + .stop_streaming = iris_vb2_stop_streaming, }; static const struct v4l2_ioctl_ops iris_v4l2_ioctl_ops = { @@ -545,6 +621,8 @@ static const struct v4l2_ioctl_ops iris_v4l2_ioctl_ops = { .vidioc_g_selection = iris_g_selection, .vidioc_subscribe_event = iris_subscribe_event, .vidioc_unsubscribe_event = iris_unsubscribe_event, + .vidioc_streamon = v4l2_m2m_ioctl_streamon, + .vidioc_streamoff = v4l2_m2m_ioctl_streamoff, }; void iris_init_ops(struct iris_core *core) From patchwork Tue Aug 27 10:05:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 822721 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E9B81A0706; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; cv=none; b=gA1gRpNlPof8rj1V4FZk4q8khT8pMfzrT644R6+QDx2PATDgJyx2ycu7d5qbSwrDxR9171Lv0GU3fnlhHAoROCOa4wLF79/rZC0IjOjG4+HhaABxmLCDXL1iQ6lzqvUkd8cDsq4rKRtf53epZtiQ9iaSQT+3ME/pRmcLoeC3dQc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; c=relaxed/simple; bh=c+mODD11bC791TS5W+5fo8kx6Ce12up39OjaztKZxIE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=sm3DxZNyP+9J1qkQgXYWifnoBVjKe/m2i0CKrFb0P5ydRNtlHq/vglZKjadqxSgqmDC0v3X1Nqmfy2CRcru5hh1OAiqSwOuqyya+MnS+ipU0oGn4VahXLax/Y1vZ/VqhcDn8rbj+C5czyI3gEEBdO2Acd7SbfHW/g8mHt4z+z08= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lmcs6Mb6; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lmcs6Mb6" Received: by smtp.kernel.org (Postfix) with ESMTPS id 08AE9C4FE0F; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753196; bh=c+mODD11bC791TS5W+5fo8kx6Ce12up39OjaztKZxIE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=lmcs6Mb6fqZgHtyBk0T9XmGW1jD7UN4se9b3jfqxCYBRSfWNhOpYxtLd7Eql5sd2N YASrkod8vs6lX6MGVtiam8Q9LmCbbRoryQiwqoiaKmBHhs1L0GoayprK/fdkw6hh5v P5SYbz32cq51C4xvcF5lDp7r81h94g4PLRFGMzSZiNQfXwQ2zQfG0vwyVm6DRfrJao 9cjGxTVjNHVo2EDQhgKFfQZt1zOUmYVaWDR6V9GzvyeXPgJ7HQBfU/j/Ub/FLBGfAz L9P0Y6i0mNKBH1M/rwOMEKvgr4zxOOWV3evZOnFDettaKTvbq5IxnH3SxWG+YkNp/g +WiyZGzp8Gjgw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id E740DC54734; Tue, 27 Aug 2024 10:06:35 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:44 +0530 Subject: [PATCH v3 19/29] media: iris: implement set properties to firmware during streamon Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-19-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal , Vedang Nagar X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=65524; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=bZG8lCDkPnbtVVaShtDKWAhdQyxMle41g23d+8+BUPQ=; b=SS/q4qkm+tkmRmzMOvS0cz6q5wE16vCZi46Cm+ao//2mrHcB20cRC2BCQtiwjQJXkPVZHc2oA aHqaRBSaOUSDJKhqsyv1VYCwYaelSzp+uZQvkWGKUcljExVMN/drb4t X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Vedang Nagar During stream on, set some mandatory properties to firmware to start a session. Set all v4l2 properties set by client, to firmware prepared with the dependency graph. Signed-off-by: Vedang Nagar Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/iris_buffer.c | 113 ++++++ drivers/media/platform/qcom/iris/iris_ctrls.c | 99 +++++ drivers/media/platform/qcom/iris/iris_ctrls.h | 7 + drivers/media/platform/qcom/iris/iris_hfi_common.h | 76 ++++ .../platform/qcom/iris/iris_hfi_gen1_command.c | 418 +++++++++++++++++++++ .../platform/qcom/iris/iris_hfi_gen1_defines.h | 79 ++++ drivers/media/platform/qcom/iris/iris_hfi_gen2.h | 2 + .../platform/qcom/iris/iris_hfi_gen2_command.c | 327 ++++++++++++++++ .../platform/qcom/iris/iris_hfi_gen2_defines.h | 20 + .../platform/qcom/iris/iris_hfi_gen2_packet.c | 112 ++++++ .../platform/qcom/iris/iris_hfi_gen2_packet.h | 7 + .../platform/qcom/iris/iris_hfi_gen2_response.c | 50 +++ .../platform/qcom/iris/iris_platform_common.h | 11 + .../platform/qcom/iris/iris_platform_sm8250.c | 19 + .../platform/qcom/iris/iris_platform_sm8550.c | 31 ++ drivers/media/platform/qcom/iris/iris_utils.c | 22 ++ drivers/media/platform/qcom/iris/iris_utils.h | 3 + drivers/media/platform/qcom/iris/iris_vdec.c | 11 + drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 22 ++ drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 1 + 20 files changed, 1430 insertions(+) diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c b/drivers/media/platform/qcom/iris/iris_buffer.c index 652117a19b45..b90b912d73b6 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/iris/iris_buffer.c @@ -64,6 +64,117 @@ static u32 iris_output_buffer_size_nv12(struct iris_inst *inst) return ALIGN(y_plane + uv_plane, PIXELS_4K); } +/* + * QC08C: + * Compressed Macro-tile format for NV12. + * Contains 4 planes in the following order - + * (A) Y_Meta_Plane + * (B) Y_UBWC_Plane + * (C) UV_Meta_Plane + * (D) UV_UBWC_Plane + * + * Y_Meta_Plane consists of meta information to decode compressed + * tile data in Y_UBWC_Plane. + * Y_UBWC_Plane consists of Y data in compressed macro-tile format. + * UBWC decoder block will use the Y_Meta_Plane data together with + * Y_UBWC_Plane data to produce loss-less uncompressed 8 bit Y samples. + * + * UV_Meta_Plane consists of meta information to decode compressed + * tile data in UV_UBWC_Plane. + * UV_UBWC_Plane consists of UV data in compressed macro-tile format. + * UBWC decoder block will use UV_Meta_Plane data together with + * UV_UBWC_Plane data to produce loss-less uncompressed 8 bit 2x2 + * subsampled color difference samples. + * + * Each tile in Y_UBWC_Plane/UV_UBWC_Plane is independently decodable + * and randomly accessible. There is no dependency between tiles. + * + * <----- y_meta_stride ----> + * <-------- Width ------> + * M M M M M M M M M M M M . . ^ ^ + * M M M M M M M M M M M M . . | | + * M M M M M M M M M M M M . . Height | + * M M M M M M M M M M M M . . | y_meta_scanlines + * M M M M M M M M M M M M . . | | + * M M M M M M M M M M M M . . | | + * M M M M M M M M M M M M . . | | + * M M M M M M M M M M M M . . V | + * . . . . . . . . . . . . . . | + * . . . . . . . . . . . . . . | + * . . . . . . . . . . . . . . -------> Buffer size aligned to 4k + * . . . . . . . . . . . . . . V + * <--Compressed tile y_stride---> + * <------- Width -------> + * Y* Y* Y* Y* Y* Y* Y* Y* . . . . ^ ^ + * Y* Y* Y* Y* Y* Y* Y* Y* . . . . | | + * Y* Y* Y* Y* Y* Y* Y* Y* . . . . Height | + * Y* Y* Y* Y* Y* Y* Y* Y* . . . . | Macro_tile y_scanlines + * Y* Y* Y* Y* Y* Y* Y* Y* . . . . | | + * Y* Y* Y* Y* Y* Y* Y* Y* . . . . | | + * Y* Y* Y* Y* Y* Y* Y* Y* . . . . | | + * Y* Y* Y* Y* Y* Y* Y* Y* . . . . V | + * . . . . . . . . . . . . . . . . | + * . . . . . . . . . . . . . . . . | + * . . . . . . . . . . . . . . . . -------> Buffer size aligned to 4k + * . . . . . . . . . . . . . . . . V + * <----- uv_meta_stride ----> + * M M M M M M M M M M M M . . ^ + * M M M M M M M M M M M M . . | + * M M M M M M M M M M M M . . | + * M M M M M M M M M M M M . . uv_meta_scanlines + * . . . . . . . . . . . . . . | + * . . . . . . . . . . . . . . V + * . . . . . . . . . . . . . . -------> Buffer size aligned to 4k + * <--Compressed tile uv_stride---> + * U* V* U* V* U* V* U* V* . . . . ^ + * U* V* U* V* U* V* U* V* . . . . | + * U* V* U* V* U* V* U* V* . . . . | + * U* V* U* V* U* V* U* V* . . . . uv_scanlines + * . . . . . . . . . . . . . . . . | + * . . . . . . . . . . . . . . . . V + * . . . . . . . . . . . . . . . . -------> Buffer size aligned to 4k + * + * y_stride = align(Width, 128) + * uv_stride = align(Width, 128) + * y_scanlines = align(Height, 32) + * uv_scanlines = align(Height/2, 16) + * y_plane = align(y_stride * y_scanlines, 4096) + * uv_plane = align(uv_stride * uv_scanlines, 4096) + * y_meta_stride = align(roundup(Width, Y_TileWidth), 64) + * y_meta_scanlines = align(roundup(Height, Y_TileHeight), 16) + * y_meta_plane = align(y_meta_stride * y_meta_scanlines, 4096) + * uv_meta_stride = align(roundup(Width, UV_TileWidth), 64) + * uv_meta_scanlines = align(roundup(Height, UV_TileHeight), 16) + * uv_meta_plane = align(uv_meta_stride * uv_meta_scanlines, 4096) + * + * Total size = align( y_plane + uv_plane + + * y_meta_plane + uv_meta_plane, 4096) + */ +static u32 iris_output_buffer_size_qc08c(struct iris_inst *inst) +{ + u32 y_plane, uv_plane, y_stride, uv_stride; + u32 uv_meta_stride, uv_meta_plane; + u32 y_meta_stride, y_meta_plane; + struct v4l2_format *f; + + f = inst->fmt_dst; + y_meta_stride = ALIGN(DIV_ROUND_UP(f->fmt.pix_mp.width, 32), 64); + y_meta_plane = y_meta_stride * ALIGN(DIV_ROUND_UP(f->fmt.pix_mp.height, 8), 16); + y_meta_plane = ALIGN(y_meta_plane, PIXELS_4K); + + y_stride = ALIGN(f->fmt.pix_mp.width, 128); + y_plane = ALIGN(y_stride * ALIGN(f->fmt.pix_mp.height, 32), PIXELS_4K); + + uv_meta_stride = ALIGN(DIV_ROUND_UP(f->fmt.pix_mp.width / 2, 16), 64); + uv_meta_plane = uv_meta_stride * ALIGN(DIV_ROUND_UP(f->fmt.pix_mp.height / 2, 8), 16); + uv_meta_plane = ALIGN(uv_meta_plane, PIXELS_4K); + + uv_stride = ALIGN(f->fmt.pix_mp.width, 128); + uv_plane = ALIGN(uv_stride * ALIGN(f->fmt.pix_mp.height / 2, 32), PIXELS_4K); + + return ALIGN(y_meta_plane + y_plane + uv_meta_plane + uv_plane, PIXELS_4K); +} + static u32 iris_input_buffer_size(struct iris_inst *inst) { u32 base_res_mbs = NUM_MBS_4K; @@ -97,6 +208,8 @@ int iris_get_buffer_size(struct iris_inst *inst, return iris_input_buffer_size(inst); case BUF_OUTPUT: return iris_output_buffer_size_nv12(inst); + case BUF_DPB: + return iris_output_buffer_size_qc08c(inst); default: return 0; } diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.c b/drivers/media/platform/qcom/iris/iris_ctrls.c index 868306d68a87..64b2f19c2b03 100644 --- a/drivers/media/platform/qcom/iris/iris_ctrls.c +++ b/drivers/media/platform/qcom/iris/iris_ctrls.c @@ -3,6 +3,8 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include + #include "iris_ctrls.h" #include "iris_instance.h" @@ -192,3 +194,100 @@ int iris_session_init_caps(struct iris_core *core) return 0; } + +static u32 iris_get_port_info(struct iris_inst *inst, + enum platform_inst_fw_cap_type cap_id) +{ + if (inst->fw_cap[cap_id].flags & CAP_FLAG_INPUT_PORT) + return HFI_PORT_BITSTREAM; + else if (inst->fw_cap[cap_id].flags & CAP_FLAG_OUTPUT_PORT) + return HFI_PORT_RAW; + + return HFI_PORT_NONE; +} + +int iris_set_u32_enum(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id) +{ + const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + u32 hfi_value = inst->fw_cap[cap_id].value; + u32 hfi_id = inst->fw_cap[cap_id].hfi_id; + + return hfi_ops->session_set_property(inst, hfi_id, + HFI_HOST_FLAGS_NONE, + iris_get_port_info(inst, cap_id), + HFI_PAYLOAD_U32_ENUM, + &hfi_value, sizeof(u32)); +} + +int iris_set_u32(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id) +{ + const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + u32 hfi_value = inst->fw_cap[cap_id].value; + u32 hfi_id = inst->fw_cap[cap_id].hfi_id; + + return hfi_ops->session_set_property(inst, hfi_id, + HFI_HOST_FLAGS_NONE, + iris_get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &hfi_value, sizeof(u32)); +} + +int iris_set_stage(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id) +{ + const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + struct v4l2_format *inp_f; + u32 work_mode = STAGE_2; + u32 width, height; + u32 hfi_id; + + hfi_id = inst->fw_cap[cap_id].hfi_id; + + inp_f = inst->fmt_src; + height = inp_f->fmt.pix_mp.height; + width = inp_f->fmt.pix_mp.width; + if (iris_res_is_less_than(width, height, 1280, 720)) + work_mode = STAGE_1; + + return hfi_ops->session_set_property(inst, hfi_id, + HFI_HOST_FLAGS_NONE, + iris_get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &work_mode, sizeof(u32)); +} + +int iris_set_pipe(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id) +{ + const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + u32 work_route, hfi_id; + + work_route = inst->fw_cap[PIPE].value; + hfi_id = inst->fw_cap[cap_id].hfi_id; + + return hfi_ops->session_set_property(inst, hfi_id, + HFI_HOST_FLAGS_NONE, + iris_get_port_info(inst, cap_id), + HFI_PAYLOAD_U32, + &work_route, sizeof(u32)); +} + +int iris_set_properties(struct iris_inst *inst, u32 plane) +{ + const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + struct platform_inst_fw_cap *cap; + int ret, i; + + ret = hfi_ops->session_set_config_params(inst, plane); + if (ret) + return ret; + + for (i = 1; i < INST_FW_CAP_MAX; i++) { + cap = &inst->fw_cap[i]; + if (!iris_valid_cap_id(cap->cap_id)) + continue; + + if (cap->cap_id && cap->set) + cap->set(inst, i); + } + + return ret; +} diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.h b/drivers/media/platform/qcom/iris/iris_ctrls.h index 46e1da847aa8..ece28623070b 100644 --- a/drivers/media/platform/qcom/iris/iris_ctrls.h +++ b/drivers/media/platform/qcom/iris/iris_ctrls.h @@ -6,10 +6,17 @@ #ifndef _IRIS_CTRLS_H_ #define _IRIS_CTRLS_H_ +#include "iris_platform_common.h" + struct iris_core; struct iris_inst; int iris_ctrls_init(struct iris_inst *inst); int iris_session_init_caps(struct iris_core *core); +int iris_set_u32_enum(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id); +int iris_set_stage(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id); +int iris_set_pipe(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id); +int iris_set_u32(struct iris_inst *inst, enum platform_inst_fw_cap_type cap_id); +int iris_set_properties(struct iris_inst *inst, u32 plane); #endif diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.h b/drivers/media/platform/qcom/iris/iris_hfi_common.h index 4ac97692d072..fa409a9b3f04 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_common.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_common.h @@ -43,11 +43,75 @@ enum hfi_packet_host_flags { HFI_HOST_FLAGS_GET_PROPERTY = 0x00000008, }; +enum hfi_color_primaries { + HFI_PRIMARIES_RESERVED = 0, + HFI_PRIMARIES_BT709 = 1, + HFI_PRIMARIES_UNSPECIFIED = 2, + HFI_PRIMARIES_BT470_SYSTEM_M = 4, + HFI_PRIMARIES_BT470_SYSTEM_BG = 5, + HFI_PRIMARIES_BT601_525 = 6, + HFI_PRIMARIES_SMPTE_ST240M = 7, + HFI_PRIMARIES_GENERIC_FILM = 8, + HFI_PRIMARIES_BT2020 = 9, + HFI_PRIMARIES_SMPTE_ST428_1 = 10, + HFI_PRIMARIES_SMPTE_RP431_2 = 11, + HFI_PRIMARIES_SMPTE_EG431_1 = 12, + HFI_PRIMARIES_SMPTE_EBU_TECH = 22, +}; + +enum hfi_transfer_characteristics { + HFI_TRANSFER_RESERVED = 0, + HFI_TRANSFER_BT709 = 1, + HFI_TRANSFER_UNSPECIFIED = 2, + HFI_TRANSFER_BT470_SYSTEM_M = 4, + HFI_TRANSFER_BT470_SYSTEM_BG = 5, + HFI_TRANSFER_BT601_525_OR_625 = 6, + HFI_TRANSFER_SMPTE_ST240M = 7, + HFI_TRANSFER_LINEAR = 8, + HFI_TRANSFER_LOG_100_1 = 9, + HFI_TRANSFER_LOG_SQRT = 10, + HFI_TRANSFER_XVYCC = 11, + HFI_TRANSFER_BT1361_0 = 12, + HFI_TRANSFER_SRGB_SYCC = 13, + HFI_TRANSFER_BT2020_14 = 14, + HFI_TRANSFER_BT2020_15 = 15, + HFI_TRANSFER_SMPTE_ST2084_PQ = 16, + HFI_TRANSFER_SMPTE_ST428_1 = 17, + HFI_TRANSFER_BT2100_2_HLG = 18, +}; + +enum hfi_matrix_coefficients { + HFI_MATRIX_COEFF_SRGB_SMPTE_ST428_1 = 0, + HFI_MATRIX_COEFF_BT709 = 1, + HFI_MATRIX_COEFF_UNSPECIFIED = 2, + HFI_MATRIX_COEFF_RESERVED = 3, + HFI_MATRIX_COEFF_FCC_TITLE_47 = 4, + HFI_MATRIX_COEFF_BT470_SYS_BG_OR_BT601_625 = 5, + HFI_MATRIX_COEFF_BT601_525_BT1358_525_OR_625 = 6, + HFI_MATRIX_COEFF_SMPTE_ST240 = 7, + HFI_MATRIX_COEFF_YCGCO = 8, + HFI_MATRIX_COEFF_BT2020_NON_CONSTANT = 9, + HFI_MATRIX_COEFF_BT2020_CONSTANT = 10, + HFI_MATRIX_COEFF_SMPTE_ST2085 = 11, + HFI_MATRIX_COEFF_SMPTE_CHROM_DERV_NON_CONSTANT = 12, + HFI_MATRIX_COEFF_SMPTE_CHROM_DERV_CONSTANT = 13, + HFI_MATRIX_COEFF_BT2100 = 14, +}; + +struct iris_hfi_prop_type_handle { + u32 type; + int (*handle)(struct iris_inst *inst); +}; + struct iris_hfi_command_ops { int (*sys_init)(struct iris_core *core); int (*sys_image_version)(struct iris_core *core); int (*sys_interframe_powercollapse)(struct iris_core *core); int (*sys_pc_prep)(struct iris_core *core); + int (*session_set_config_params)(struct iris_inst *inst, u32 plane); + int (*session_set_property)(struct iris_inst *inst, + u32 packet_type, u32 flag, u32 plane, u32 payload_type, + void *payload, u32 payload_size); int (*session_open)(struct iris_inst *inst); int (*session_start)(struct iris_inst *inst, u32 plane); int (*session_stop)(struct iris_inst *inst, u32 plane); @@ -58,6 +122,18 @@ struct iris_hfi_response_ops { void (*hfi_response_handler)(struct iris_core *core); }; +struct hfi_subscription_params { + u32 bitstream_resolution; + u32 crop_offsets[2]; + u32 bit_depth; + u32 coded_frames; + u32 fw_min_count; + u32 pic_order_cnt; + u32 color_info; + u32 profile; + u32 level; +}; + int iris_hfi_core_init(struct iris_core *core); int iris_hfi_pm_suspend(struct iris_core *core); int iris_hfi_pm_resume(struct iris_core *core); diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c index a2e29f03bfee..65f561ec76ab 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c @@ -6,6 +6,7 @@ #include "iris_hfi_gen1.h" #include "iris_hfi_gen1_defines.h" #include "iris_instance.h" +#include "iris_vpu_buffer.h" static int iris_hfi_gen1_sys_init(struct iris_core *core) { @@ -182,12 +183,429 @@ static int iris_hfi_gen1_session_stop(struct iris_inst *inst, u32 plane) return ret; } +static int +iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *packet, + struct iris_inst *inst, u32 ptype, void *pdata) +{ + void *prop_data; + + prop_data = &packet->data[1]; + + packet->shdr.hdr.size = sizeof(*packet); + packet->shdr.hdr.pkt_type = HFI_CMD_SESSION_SET_PROPERTY; + packet->shdr.session_id = inst->session_id; + packet->num_properties = 1; + packet->data[0] = ptype; + + switch (ptype) { + case HFI_PROPERTY_PARAM_FRAME_SIZE: { + struct hfi_framesize *in = pdata, *fsize = prop_data; + + fsize->buffer_type = in->buffer_type; + fsize->height = in->height; + fsize->width = in->width; + packet->shdr.hdr.size += sizeof(u32) + sizeof(*fsize); + break; + } + case HFI_PROPERTY_CONFIG_VIDEOCORES_USAGE: { + struct hfi_videocores_usage_type *in = pdata, *cu = prop_data; + + cu->video_core_enable_mask = in->video_core_enable_mask; + packet->shdr.hdr.size += sizeof(u32) + sizeof(*cu); + break; + } + case HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SELECT: { + struct hfi_uncompressed_format_select *in = pdata; + struct hfi_uncompressed_format_select *hfi = prop_data; + + hfi->buffer_type = in->buffer_type; + hfi->format = in->format; + packet->shdr.hdr.size += sizeof(u32) + sizeof(*hfi); + break; + } + case HFI_PROPERTY_PARAM_UNCOMPRESSED_PLANE_ACTUAL_CONSTRAINTS_INFO: { + struct hfi_uncompressed_plane_actual_constraints_info *info = prop_data; + + info->buffer_type = HFI_BUFFER_OUTPUT2; + info->num_planes = 2; + info->plane_format[0].stride_multiples = 128; + info->plane_format[0].max_stride = 8192; + info->plane_format[0].min_plane_buffer_height_multiple = 32; + info->plane_format[0].buffer_alignment = 256; + if (info->num_planes > 1) { + info->plane_format[1].stride_multiples = 128; + info->plane_format[1].max_stride = 8192; + info->plane_format[1].min_plane_buffer_height_multiple = 16; + info->plane_format[1].buffer_alignment = 256; + } + + packet->shdr.hdr.size += sizeof(u32) + sizeof(*info); + break; + } + case HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL: { + struct hfi_buffer_count_actual *in = pdata; + struct hfi_buffer_count_actual *count = prop_data; + + count->count_actual = in->count_actual; + count->type = in->type; + count->count_min_host = in->count_min_host; + packet->shdr.hdr.size += sizeof(u32) + sizeof(*count); + break; + } + case HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM: { + struct hfi_multi_stream *in = pdata; + struct hfi_multi_stream *multi = prop_data; + + multi->buffer_type = in->buffer_type; + multi->enable = in->enable; + packet->shdr.hdr.size += sizeof(u32) + sizeof(*multi); + break; + } + case HFI_PROPERTY_PARAM_BUFFER_SIZE_ACTUAL: { + struct hfi_buffer_size_actual *in = pdata, *sz = prop_data; + + sz->size = in->size; + sz->type = in->type; + packet->shdr.hdr.size += sizeof(u32) + sizeof(*sz); + break; + } + case HFI_PROPERTY_PARAM_WORK_ROUTE: { + struct hfi_video_work_route *wr = prop_data; + u32 *in = pdata; + + wr->video_work_route = *in; + packet->shdr.hdr.size += sizeof(u32) + sizeof(*wr); + break; + } + case HFI_PROPERTY_PARAM_WORK_MODE: { + struct hfi_video_work_mode *wm = prop_data; + u32 *in = pdata; + + wm->video_work_mode = *in; + packet->shdr.hdr.size += sizeof(u32) + sizeof(*wm); + break; + } + case HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER: { + struct hfi_enable *en = prop_data; + u32 *in = pdata; + + en->enable = *in; + packet->shdr.hdr.size += sizeof(u32) + sizeof(*en); + break; + } + default: + return -EINVAL; + } + + return 0; +} + +static int hfi_gen1_set_property(struct iris_inst *inst, u32 packet_type, + void *payload, u32 payload_size) +{ + struct hfi_session_set_property_pkt *pkt; + u32 packet_size; + int ret; + + packet_size = sizeof(*pkt) + sizeof(u32) + payload_size; + pkt = kzalloc(packet_size, GFP_KERNEL); + if (!pkt) + return -ENOMEM; + + ret = iris_hfi_gen1_packet_session_set_property(pkt, inst, packet_type, payload); + if (ret == -EOPNOTSUPP) { + ret = 0; + goto exit; + } + if (ret) + goto exit; + + ret = iris_hfi_queue_cmd_write(inst->core, pkt, pkt->shdr.hdr.size); + +exit: + kfree(pkt); + + return ret; +} + +static int iris_hfi_gen1_session_set_property(struct iris_inst *inst, u32 packet_type, + u32 flag, u32 plane, u32 payload_type, + void *payload, u32 payload_size) +{ + return hfi_gen1_set_property(inst, packet_type, payload, payload_size); +} + +static int iris_hfi_gen1_set_resolution(struct iris_inst *inst) +{ + u32 ptype = HFI_PROPERTY_PARAM_FRAME_SIZE; + struct hfi_framesize fs; + int ret; + + fs.buffer_type = HFI_BUFFER_INPUT; + fs.width = inst->fmt_src->fmt.pix_mp.width; + fs.height = inst->fmt_src->fmt.pix_mp.height; + + ret = hfi_gen1_set_property(inst, ptype, &fs, sizeof(fs)); + if (ret) + return ret; + + fs.buffer_type = HFI_BUFFER_OUTPUT2; + fs.width = inst->fmt_dst->fmt.pix_mp.width; + fs.height = inst->fmt_dst->fmt.pix_mp.height; + + return hfi_gen1_set_property(inst, ptype, &fs, sizeof(fs)); +} + +static int iris_hfi_gen1_decide_core(struct iris_inst *inst) +{ + const u32 ptype = HFI_PROPERTY_CONFIG_VIDEOCORES_USAGE; + struct hfi_videocores_usage_type cu; + + cu.video_core_enable_mask = HFI_CORE_ID_1; + + return hfi_gen1_set_property(inst, ptype, &cu, sizeof(cu)); +} + +static int iris_hfi_gen1_set_raw_format(struct iris_inst *inst) +{ + const u32 ptype = HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SELECT; + struct hfi_uncompressed_format_select fmt; + u32 pixelformat, ret; + + pixelformat = inst->fmt_dst->fmt.pix_mp.pixelformat; + + if (iris_split_mode_enabled(inst)) { + fmt.buffer_type = HFI_BUFFER_OUTPUT; + fmt.format = pixelformat == V4L2_PIX_FMT_NV12 ? HFI_COLOR_FORMAT_NV12_UBWC : 0; + + ret = hfi_gen1_set_property(inst, ptype, &fmt, sizeof(fmt)); + if (ret) + return ret; + + fmt.buffer_type = HFI_BUFFER_OUTPUT2; + fmt.format = pixelformat == V4L2_PIX_FMT_NV12 ? HFI_COLOR_FORMAT_NV12 : 0; + + ret = hfi_gen1_set_property(inst, ptype, &fmt, sizeof(fmt)); + } else { + fmt.buffer_type = HFI_BUFFER_OUTPUT; + fmt.format = pixelformat == V4L2_PIX_FMT_NV12 ? HFI_COLOR_FORMAT_NV12 : 0; + + ret = hfi_gen1_set_property(inst, ptype, &fmt, sizeof(fmt)); + } + + return ret; +} + +static int iris_hfi_gen1_set_format_constraints(struct iris_inst *inst) +{ + const u32 ptype = HFI_PROPERTY_PARAM_UNCOMPRESSED_PLANE_ACTUAL_CONSTRAINTS_INFO; + struct hfi_uncompressed_plane_actual_constraints_info pconstraint; + + pconstraint.buffer_type = HFI_BUFFER_OUTPUT2; + pconstraint.num_planes = 2; + pconstraint.plane_format[0].stride_multiples = 128; + pconstraint.plane_format[0].max_stride = 8192; + pconstraint.plane_format[0].min_plane_buffer_height_multiple = 32; + pconstraint.plane_format[0].buffer_alignment = 256; + + pconstraint.plane_format[1].stride_multiples = 128; + pconstraint.plane_format[1].max_stride = 8192; + pconstraint.plane_format[1].min_plane_buffer_height_multiple = 16; + pconstraint.plane_format[1].buffer_alignment = 256; + + return hfi_gen1_set_property(inst, ptype, &pconstraint, sizeof(pconstraint)); +} + +static int iris_hfi_gen1_set_num_bufs(struct iris_inst *inst) +{ + u32 ptype = HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL; + struct hfi_buffer_count_actual buf_count; + int ret; + + buf_count.type = HFI_BUFFER_INPUT; + buf_count.count_actual = inst->buffers[BUF_INPUT].actual_count; + buf_count.count_min_host = inst->buffers[BUF_INPUT].actual_count; + + ret = hfi_gen1_set_property(inst, ptype, &buf_count, sizeof(buf_count)); + if (ret) + return ret; + + if (iris_split_mode_enabled(inst)) { + buf_count.type = HFI_BUFFER_OUTPUT; + buf_count.count_actual = inst->buffers[BUF_OUTPUT].actual_count; + buf_count.count_min_host = inst->buffers[BUF_OUTPUT].actual_count; + + ret = hfi_gen1_set_property(inst, ptype, &buf_count, sizeof(buf_count)); + if (ret) + return ret; + + buf_count.type = HFI_BUFFER_OUTPUT2; + buf_count.count_actual = iris_vpu_buf_count(inst, BUF_DPB); + buf_count.count_min_host = iris_vpu_buf_count(inst, BUF_DPB); + + ret = hfi_gen1_set_property(inst, ptype, &buf_count, sizeof(buf_count)); + } else { + buf_count.type = HFI_BUFFER_OUTPUT; + buf_count.count_actual = inst->buffers[BUF_OUTPUT].actual_count; + buf_count.count_min_host = inst->buffers[BUF_OUTPUT].actual_count; + + ret = hfi_gen1_set_property(inst, ptype, &buf_count, sizeof(buf_count)); + } + + return ret; +} + +static int iris_hfi_gen1_set_multistream(struct iris_inst *inst) +{ + u32 ptype = HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM; + struct hfi_multi_stream multi = {0}; + int ret; + + if (iris_split_mode_enabled(inst)) { + multi.buffer_type = HFI_BUFFER_OUTPUT; + multi.enable = 0; + + ret = hfi_gen1_set_property(inst, ptype, &multi, sizeof(multi)); + if (ret) + return ret; + + multi.buffer_type = HFI_BUFFER_OUTPUT2; + multi.enable = 1; + + ret = hfi_gen1_set_property(inst, ptype, &multi, sizeof(multi)); + } else { + multi.buffer_type = HFI_BUFFER_OUTPUT; + multi.enable = 1; + + ret = hfi_gen1_set_property(inst, ptype, &multi, sizeof(multi)); + if (ret) + return ret; + + multi.buffer_type = HFI_BUFFER_OUTPUT2; + multi.enable = 0; + + ret = hfi_gen1_set_property(inst, ptype, &multi, sizeof(multi)); + } + + return ret; +} + +static int iris_hfi_gen1_set_bufsize(struct iris_inst *inst) +{ + const u32 ptype = HFI_PROPERTY_PARAM_BUFFER_SIZE_ACTUAL; + struct hfi_buffer_size_actual bufsz; + int ret; + + if (iris_split_mode_enabled(inst)) { + bufsz.type = HFI_BUFFER_OUTPUT; + bufsz.size = iris_vpu_dec_dpb_size(inst); + + ret = hfi_gen1_set_property(inst, ptype, &bufsz, sizeof(bufsz)); + if (ret) + return ret; + + bufsz.type = HFI_BUFFER_OUTPUT2; + bufsz.size = inst->buffers[BUF_OUTPUT].size; + + ret = hfi_gen1_set_property(inst, ptype, &bufsz, sizeof(bufsz)); + } else { + bufsz.type = HFI_BUFFER_OUTPUT; + bufsz.size = inst->buffers[BUF_OUTPUT].size; + + ret = hfi_gen1_set_property(inst, ptype, &bufsz, sizeof(bufsz)); + if (ret) + return ret; + + bufsz.type = HFI_BUFFER_OUTPUT2; + bufsz.size = 0; + + ret = hfi_gen1_set_property(inst, ptype, &bufsz, sizeof(bufsz)); + } + + return ret; +} + +static int iris_hfi_gen1_session_set_config_params(struct iris_inst *inst, u32 plane) +{ + struct iris_core *core = inst->core; + const u32 *config_params; + u32 config_params_size; + int ret = 0; + u32 i, j; + + static const struct iris_hfi_prop_type_handle prop_type_handle_inp_arr[] = { + {HFI_PROPERTY_PARAM_FRAME_SIZE, + iris_hfi_gen1_set_resolution}, + {HFI_PROPERTY_CONFIG_VIDEOCORES_USAGE, + iris_hfi_gen1_decide_core}, + {HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SELECT, + iris_hfi_gen1_set_raw_format}, + {HFI_PROPERTY_PARAM_UNCOMPRESSED_PLANE_ACTUAL_CONSTRAINTS_INFO, + iris_hfi_gen1_set_format_constraints}, + {HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL, + iris_hfi_gen1_set_num_bufs}, + {HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM, + iris_hfi_gen1_set_multistream}, + {HFI_PROPERTY_PARAM_BUFFER_SIZE_ACTUAL, + iris_hfi_gen1_set_bufsize}, + }; + + static const struct iris_hfi_prop_type_handle prop_type_handle_out_arr[] = { + {HFI_PROPERTY_PARAM_FRAME_SIZE, + iris_hfi_gen1_set_resolution}, + {HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SELECT, + iris_hfi_gen1_set_raw_format}, + {HFI_PROPERTY_PARAM_UNCOMPRESSED_PLANE_ACTUAL_CONSTRAINTS_INFO, + iris_hfi_gen1_set_format_constraints}, + {HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL, + iris_hfi_gen1_set_num_bufs}, + {HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM, + iris_hfi_gen1_set_multistream}, + {HFI_PROPERTY_PARAM_BUFFER_SIZE_ACTUAL, + iris_hfi_gen1_set_bufsize}, + }; + + config_params = core->iris_platform_data->input_config_params; + config_params_size = core->iris_platform_data->input_config_params_size; + + if (!config_params || !config_params_size) + return -EINVAL; + + if (V4L2_TYPE_IS_OUTPUT(plane)) { + for (i = 0; i < config_params_size; i++) { + for (j = 0; j < ARRAY_SIZE(prop_type_handle_inp_arr); j++) { + if (prop_type_handle_inp_arr[j].type == config_params[i]) { + ret = prop_type_handle_inp_arr[j].handle(inst); + if (ret) + return ret; + break; + } + } + } + } else if (V4L2_TYPE_IS_CAPTURE(plane)) { + for (i = 0; i < config_params_size; i++) { + for (j = 0; j < ARRAY_SIZE(prop_type_handle_out_arr); j++) { + if (prop_type_handle_out_arr[j].type == config_params[i]) { + ret = prop_type_handle_out_arr[j].handle(inst); + if (ret) + return ret; + break; + } + } + } + } + + return ret; +} + static const struct iris_hfi_command_ops iris_hfi_gen1_command_ops = { .sys_init = iris_hfi_gen1_sys_init, .sys_image_version = iris_hfi_gen1_sys_image_version, .sys_interframe_powercollapse = iris_hfi_gen1_sys_interframe_powercollapse, .sys_pc_prep = iris_hfi_gen1_sys_pc_prep, .session_open = iris_hfi_gen1_session_open, + .session_set_config_params = iris_hfi_gen1_session_set_config_params, + .session_set_property = iris_hfi_gen1_session_set_property, .session_start = iris_hfi_gen1_session_start, .session_stop = iris_hfi_gen1_session_stop, .session_close = iris_hfi_gen1_session_close, diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h index fb11d3e45c9f..8b48a082bcd3 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h @@ -23,6 +23,8 @@ #define HFI_CMD_SYS_SESSION_INIT 0x10007 #define HFI_CMD_SYS_SESSION_END 0x10008 +#define HFI_CMD_SESSION_SET_PROPERTY 0x11001 + #define HFI_CMD_SESSION_LOAD_RESOURCES 0x211001 #define HFI_CMD_SESSION_START 0x211002 #define HFI_CMD_SESSION_STOP 0x211003 @@ -41,12 +43,31 @@ #define HFI_FLUSH_OUTPUT2 0x1000003 #define HFI_FLUSH_ALL 0x1000004 +#define HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL 0x201001 +#define HFI_PROPERTY_PARAM_UNCOMPRESSED_PLANE_ACTUAL_CONSTRAINTS_INFO 0x201002 +#define HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE 0x201008 +#define HFI_PROPERTY_PARAM_BUFFER_SIZE_ACTUAL 0x20100c + #define HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER 0x1200001 + +#define HFI_BUFFER_INPUT 0x1 +#define HFI_BUFFER_OUTPUT 0x2 +#define HFI_BUFFER_OUTPUT2 0x3 + #define HFI_PROPERTY_SYS_CODEC_POWER_PLANE_CTRL 0x5 #define HFI_PROPERTY_SYS_IMAGE_VERSION 0x6 +#define HFI_PROPERTY_PARAM_FRAME_SIZE 0x1001 +#define HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SELECT 0x1003 #define HFI_PROPERTY_PARAM_WORK_MODE 0x1015 #define HFI_PROPERTY_PARAM_WORK_ROUTE 0x1017 +#define HFI_PROPERTY_CONFIG_VIDEOCORES_USAGE 0x2002 + +#define HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM 0x1003001 +#define HFI_CORE_ID_1 1 +#define HFI_COLOR_FORMAT_NV12 0x02 +#define HFI_COLOR_FORMAT_NV12_UBWC 0x8002 + #define HFI_MSG_SYS_INIT 0x20001 #define HFI_MSG_SYS_SESSION_INIT 0x20006 #define HFI_MSG_SYS_SESSION_END 0x20007 @@ -97,6 +118,12 @@ struct hfi_sys_get_property_pkt { u32 data; }; +struct hfi_session_set_property_pkt { + struct hfi_session_hdr_pkt shdr; + u32 num_properties; + u32 data[]; +}; + struct hfi_sys_pc_prep_pkt { struct hfi_pkt_hdr hdr; }; @@ -148,6 +175,58 @@ struct hfi_enable { u32 enable; }; +struct hfi_framesize { + u32 buffer_type; + u32 width; + u32 height; +}; + +struct hfi_videocores_usage_type { + u32 video_core_enable_mask; +}; + +struct hfi_video_work_mode { + u32 video_work_mode; +}; + +struct hfi_video_work_route { + u32 video_work_route; +}; + +struct hfi_uncompressed_format_select { + u32 buffer_type; + u32 format; +}; + +struct hfi_uncompressed_plane_constraints { + u32 stride_multiples; + u32 max_stride; + u32 min_plane_buffer_height_multiple; + u32 buffer_alignment; +}; + +struct hfi_uncompressed_plane_actual_constraints_info { + u32 buffer_type; + u32 num_planes; + struct hfi_uncompressed_plane_constraints plane_format[2]; +}; + +struct hfi_buffer_count_actual { + u32 type; + u32 count_actual; + u32 count_min_host; +}; + +struct hfi_buffer_size_actual { + u32 type; + u32 size; +}; + +struct hfi_multi_stream { + u32 buffer_type; + u32 enable; +}; + struct hfi_msg_sys_debug_pkt { struct hfi_pkt_hdr hdr; u32 msg_type; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2.h index 76f0c9032a92..8170c1fef569 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2.h @@ -18,10 +18,12 @@ struct iris_core; * * @inst: pointer to iris_instance structure * @packet: HFI packet + * @src_subcr_params: subscription params to fw on input port */ struct iris_inst_hfi_gen2 { struct iris_inst inst; struct iris_hfi_header *packet; + struct hfi_subscription_params src_subcr_params; }; void iris_hfi_gen2_command_ops_init(struct iris_core *core); diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c index 5ace377fec7a..e50f00021f6d 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c @@ -3,9 +3,12 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include + #include "iris_hfi_gen2.h" #include "iris_hfi_gen2_packet.h" +#define UNSPECIFIED_COLOR_FORMAT 5 #define NUM_SYS_INIT_PACKETS 8 static int iris_hfi_gen2_sys_init(struct iris_core *core) @@ -103,6 +106,328 @@ static u32 iris_hfi_gen2_get_port(u32 plane) return hfi_port; } +static int iris_hfi_gen2_session_set_property(struct iris_inst *inst, u32 packet_type, u32 flag, + u32 plane, u32 payload_type, void *payload, + u32 payload_size) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + + iris_hfi_gen2_packet_session_property(inst, + packet_type, + flag, + plane, + payload_type, + payload, + payload_size); + + return iris_hfi_queue_cmd_write(inst->core, inst_hfi_gen2->packet, + inst_hfi_gen2->packet->size); +} + +static int iris_hfi_gen2_set_bitstream_resolution(struct iris_inst *inst) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + u32 resolution; + + resolution = inst->fmt_src->fmt.pix_mp.width << 16 | + inst->fmt_src->fmt.pix_mp.height; + inst_hfi_gen2->src_subcr_params.bitstream_resolution = resolution; + + return + iris_hfi_gen2_session_set_property(inst, + HFI_PROP_BITSTREAM_RESOLUTION, + HFI_HOST_FLAGS_NONE, + port, + HFI_PAYLOAD_U32, + &resolution, + sizeof(u32)); +} + +static int iris_hfi_gen2_set_crop_offsets(struct iris_inst *inst) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + u32 left_offset, top_offset, right_offset, bottom_offset; + u32 payload[2] = {0}; + + left_offset = inst->crop.left; + top_offset = inst->crop.top; + right_offset = (inst->fmt_src->fmt.pix_mp.width - + inst->crop.width); + bottom_offset = (inst->fmt_src->fmt.pix_mp.height - + inst->crop.height); + + payload[0] = FIELD_PREP(GENMASK(31, 16), left_offset) | top_offset; + payload[1] = FIELD_PREP(GENMASK(31, 16), right_offset) | bottom_offset; + inst_hfi_gen2->src_subcr_params.crop_offsets[0] = payload[0]; + inst_hfi_gen2->src_subcr_params.crop_offsets[1] = payload[1]; + + return + iris_hfi_gen2_session_set_property(inst, + HFI_PROP_CROP_OFFSETS, + HFI_HOST_FLAGS_NONE, + port, + HFI_PAYLOAD_64_PACKED, + &payload, + sizeof(u64)); +} + +static int iris_hfi_gen2_set_bit_dpeth(struct iris_inst *inst) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + u32 bitdepth = BIT_DEPTH_8; + + inst_hfi_gen2->src_subcr_params.bit_depth = bitdepth; + + return + iris_hfi_gen2_session_set_property(inst, + HFI_PROP_LUMA_CHROMA_BIT_DEPTH, + HFI_HOST_FLAGS_NONE, + port, + HFI_PAYLOAD_U32, + &bitdepth, + sizeof(u32)); +} + +static int iris_hfi_gen2_set_coded_frames(struct iris_inst *inst) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + u32 coded_frames = 0; + + if (inst->fw_cap[CODED_FRAMES].value == CODED_FRAMES_PROGRESSIVE) + coded_frames = HFI_BITMASK_FRAME_MBS_ONLY_FLAG; + inst_hfi_gen2->src_subcr_params.coded_frames = coded_frames; + + return + iris_hfi_gen2_session_set_property(inst, + HFI_PROP_CODED_FRAMES, + HFI_HOST_FLAGS_NONE, + port, + HFI_PAYLOAD_U32, + &coded_frames, + sizeof(u32)); +} + +static int iris_hfi_gen2_set_min_output_count(struct iris_inst *inst) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + u32 min_output; + + min_output = inst->buffers[BUF_OUTPUT].min_count; + inst_hfi_gen2->src_subcr_params.fw_min_count = min_output; + + return + iris_hfi_gen2_session_set_property(inst, + HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT, + HFI_HOST_FLAGS_NONE, + port, + HFI_PAYLOAD_U32, + &min_output, + sizeof(u32)); +} + +static int iris_hfi_gen2_set_picture_order_count(struct iris_inst *inst) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + u32 poc = 0; + + inst_hfi_gen2->src_subcr_params.pic_order_cnt = poc; + + return + iris_hfi_gen2_session_set_property(inst, + HFI_PROP_PIC_ORDER_CNT_TYPE, + HFI_HOST_FLAGS_NONE, + port, + HFI_PAYLOAD_U32, + &poc, + sizeof(u32)); +} + +static int iris_hfi_gen2_set_colorspace(struct iris_inst *inst) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + u32 video_signal_type_present_flag = 0, color_info = 0; + u32 matrix_coeff = HFI_MATRIX_COEFF_RESERVED; + u32 video_format = UNSPECIFIED_COLOR_FORMAT; + struct v4l2_pix_format_mplane *pixmp = NULL; + u32 full_range = V4L2_QUANTIZATION_DEFAULT; + u32 transfer_char = HFI_TRANSFER_RESERVED; + u32 colour_description_present_flag = 0; + u32 primaries = HFI_PRIMARIES_RESERVED; + + pixmp = &inst->fmt_src->fmt.pix_mp; + if (pixmp->colorspace != V4L2_COLORSPACE_DEFAULT || + pixmp->ycbcr_enc != V4L2_YCBCR_ENC_DEFAULT || + pixmp->xfer_func != V4L2_XFER_FUNC_DEFAULT) { + colour_description_present_flag = 1; + video_signal_type_present_flag = 1; + primaries = iris_hfi_gen2_get_color_primaries(pixmp->colorspace); + matrix_coeff = iris_hfi_gen2_get_matrix_coefficients(pixmp->ycbcr_enc); + transfer_char = iris_hfi_gen2_get_transfer_char(pixmp->xfer_func); + } + + if (pixmp->quantization != V4L2_QUANTIZATION_DEFAULT) { + video_signal_type_present_flag = 1; + full_range = pixmp->quantization == + V4L2_QUANTIZATION_FULL_RANGE ? 1 : 0; + } + + color_info = iris_hfi_gen2_get_color_info(matrix_coeff, transfer_char, primaries, + colour_description_present_flag, full_range, + video_format, video_signal_type_present_flag); + + inst_hfi_gen2->src_subcr_params.color_info = color_info; + + return + iris_hfi_gen2_session_set_property(inst, + HFI_PROP_SIGNAL_COLOR_INFO, + HFI_HOST_FLAGS_NONE, + port, + HFI_PAYLOAD_32_PACKED, + &color_info, + sizeof(u32)); +} + +static int iris_hfi_gen2_set_profile(struct iris_inst *inst) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + u32 profile; + + profile = inst->fw_cap[PROFILE].value; + inst_hfi_gen2->src_subcr_params.profile = profile; + + return + iris_hfi_gen2_session_set_property(inst, + HFI_PROP_PROFILE, + HFI_HOST_FLAGS_NONE, + port, + HFI_PAYLOAD_U32_ENUM, + &profile, + sizeof(u32)); +} + +static int iris_hfi_gen2_set_level(struct iris_inst *inst) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + u32 level; + + level = inst->fw_cap[LEVEL].value; + inst_hfi_gen2->src_subcr_params.level = level; + + return + iris_hfi_gen2_session_set_property(inst, + HFI_PROP_LEVEL, + HFI_HOST_FLAGS_NONE, + port, + HFI_PAYLOAD_U32_ENUM, + &level, + sizeof(u32)); +} + +static int iris_hfi_gen2_set_colorformat(struct iris_inst *inst) +{ + u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + u32 hfi_colorformat; + u32 pixelformat; + + pixelformat = inst->fmt_dst->fmt.pix_mp.pixelformat; + hfi_colorformat = pixelformat == V4L2_PIX_FMT_NV12 ? HFI_COLOR_FMT_NV12 : 0; + + return + iris_hfi_gen2_session_set_property(inst, + HFI_PROP_COLOR_FORMAT, + HFI_HOST_FLAGS_NONE, + port, + HFI_PAYLOAD_U32, + &hfi_colorformat, + sizeof(u32)); +} + +static int iris_hfi_gen2_set_linear_stride_scanline(struct iris_inst *inst) +{ + u32 port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + u32 stride_y, scanline_y, stride_uv, scanline_uv; + u32 pixelformat; + u32 payload[2]; + + pixelformat = inst->fmt_dst->fmt.pix_mp.pixelformat; + + if (pixelformat != V4L2_PIX_FMT_NV12) + return 0; + + stride_y = inst->fmt_dst->fmt.pix_mp.width; + scanline_y = inst->fmt_dst->fmt.pix_mp.height; + stride_uv = stride_y; + scanline_uv = scanline_y / 2; + + payload[0] = stride_y << 16 | scanline_y; + payload[1] = stride_uv << 16 | scanline_uv; + + return + iris_hfi_gen2_session_set_property(inst, + HFI_PROP_LINEAR_STRIDE_SCANLINE, + HFI_HOST_FLAGS_NONE, + port, + HFI_PAYLOAD_U64, + &payload, + sizeof(u64)); +} + +static int iris_hfi_gen2_session_set_config_params(struct iris_inst *inst, u32 plane) +{ + struct iris_core *core = inst->core; + const u32 *config_params; + u32 config_params_size; + int ret = 0; + u32 i, j; + + static const struct iris_hfi_prop_type_handle prop_type_handle_arr[] = { + {HFI_PROP_BITSTREAM_RESOLUTION, iris_hfi_gen2_set_bitstream_resolution }, + {HFI_PROP_CROP_OFFSETS, iris_hfi_gen2_set_crop_offsets }, + {HFI_PROP_CODED_FRAMES, iris_hfi_gen2_set_coded_frames }, + {HFI_PROP_LUMA_CHROMA_BIT_DEPTH, iris_hfi_gen2_set_bit_dpeth }, + {HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT, iris_hfi_gen2_set_min_output_count }, + {HFI_PROP_PIC_ORDER_CNT_TYPE, iris_hfi_gen2_set_picture_order_count }, + {HFI_PROP_SIGNAL_COLOR_INFO, iris_hfi_gen2_set_colorspace }, + {HFI_PROP_PROFILE, iris_hfi_gen2_set_profile }, + {HFI_PROP_LEVEL, iris_hfi_gen2_set_level }, + {HFI_PROP_COLOR_FORMAT, iris_hfi_gen2_set_colorformat }, + {HFI_PROP_LINEAR_STRIDE_SCANLINE, iris_hfi_gen2_set_linear_stride_scanline }, + }; + + if (V4L2_TYPE_IS_OUTPUT(plane)) { + config_params = core->iris_platform_data->input_config_params; + config_params_size = core->iris_platform_data->input_config_params_size; + } else { + config_params = core->iris_platform_data->output_config_params; + config_params_size = core->iris_platform_data->output_config_params_size; + } + + if (!config_params || !config_params_size) + return -EINVAL; + + for (i = 0; i < config_params_size; i++) { + for (j = 0; j < ARRAY_SIZE(prop_type_handle_arr); j++) { + if (prop_type_handle_arr[j].type == config_params[i]) { + ret = prop_type_handle_arr[j].handle(inst); + if (ret) + return ret; + break; + } + } + } + + return ret; +} + static int iris_hfi_gen2_session_set_codec(struct iris_inst *inst) { struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); @@ -261,6 +586,8 @@ static const struct iris_hfi_command_ops iris_hfi_gen2_command_ops = { .sys_interframe_powercollapse = iris_hfi_gen2_sys_interframe_powercollapse, .sys_pc_prep = iris_hfi_gen2_sys_pc_prep, .session_open = iris_hfi_gen2_session_open, + .session_set_config_params = iris_hfi_gen2_session_set_config_params, + .session_set_property = iris_hfi_gen2_session_set_property, .session_start = iris_hfi_gen2_session_start, .session_stop = iris_hfi_gen2_session_stop, .session_close = iris_hfi_gen2_session_close, diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h index f822b91bc217..b5b232d31ad8 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h @@ -19,6 +19,8 @@ #define HFI_CMD_STOP 0x01000006 #define HFI_CMD_END 0x01FFFFFF +#define HFI_BITMASK_FRAME_MBS_ONLY_FLAG 0x00000001 + #define HFI_PROP_BEGIN 0x03000000 #define HFI_PROP_IMAGE_VERSION 0x03000001 #define HFI_PROP_INTRA_FRAME_POWER_COLLAPSE 0x03000002 @@ -30,6 +32,10 @@ #define HFI_PROP_UBWC_BANK_SWZL_LEVEL3 0x03000008 #define HFI_PROP_UBWC_BANK_SPREADING 0x03000009 #define HFI_PROP_CODEC 0x03000100 +#define HFI_PROP_COLOR_FORMAT 0x03000101 +#define HFI_PROP_BITSTREAM_RESOLUTION 0x03000103 +#define HFI_PROP_LINEAR_STRIDE_SCANLINE 0x03000104 +#define HFI_PROP_CROP_OFFSETS 0x03000105 #define HFI_PROP_PROFILE 0x03000107 #define HFI_PROP_LEVEL 0x03000108 #define HFI_PROP_STAGE 0x0300010a @@ -37,7 +43,10 @@ #define HFI_PROP_LUMA_CHROMA_BIT_DEPTH 0x0300010f #define HFI_PROP_CODED_FRAMES 0x03000120 #define HFI_PROP_BUFFER_HOST_MAX_COUNT 0x03000123 +#define HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT 0x03000124 #define HFI_PROP_PIC_ORDER_CNT_TYPE 0x03000128 +#define HFI_PROP_QUALITY_MODE 0x03000148 +#define HFI_PROP_SIGNAL_COLOR_INFO 0x03000155 #define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168 #define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169 #define HFI_PROP_END 0x03FFFFFF @@ -55,6 +64,17 @@ #define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001 #define HFI_SYSTEM_ERROR_END 0x05FFFFFF +enum hfi_color_format { + HFI_COLOR_FMT_OPAQUE = 0, + HFI_COLOR_FMT_NV12 = 1, + HFI_COLOR_FMT_NV12_UBWC = 2, + HFI_COLOR_FMT_P010 = 3, + HFI_COLOR_FMT_TP10_UBWC = 4, + HFI_COLOR_FMT_RGBA8888 = 5, + HFI_COLOR_FMT_RGBA8888_UBWC = 6, + HFI_COLOR_FMT_NV21 = 7, +}; + enum hfi_codec_type { HFI_CODEC_DECODE_AVC = 1, HFI_CODEC_ENCODE_AVC = 2, diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c index 09e7c07fdc5f..33788063f487 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.c @@ -7,6 +7,118 @@ #include "iris_hfi_gen2.h" #include "iris_hfi_gen2_packet.h" +u32 iris_hfi_gen2_get_color_primaries(u32 primaries) +{ + u32 hfi_primaries = HFI_PRIMARIES_RESERVED; + + switch (primaries) { + case V4L2_COLORSPACE_DEFAULT: + hfi_primaries = HFI_PRIMARIES_RESERVED; + break; + case V4L2_COLORSPACE_REC709: + hfi_primaries = HFI_PRIMARIES_BT709; + break; + case V4L2_COLORSPACE_470_SYSTEM_M: + hfi_primaries = HFI_PRIMARIES_BT470_SYSTEM_M; + break; + case V4L2_COLORSPACE_470_SYSTEM_BG: + hfi_primaries = HFI_PRIMARIES_BT470_SYSTEM_BG; + break; + case V4L2_COLORSPACE_SMPTE170M: + hfi_primaries = HFI_PRIMARIES_BT601_525; + break; + case V4L2_COLORSPACE_SMPTE240M: + hfi_primaries = HFI_PRIMARIES_SMPTE_ST240M; + break; + case V4L2_COLORSPACE_BT2020: + hfi_primaries = HFI_PRIMARIES_BT2020; + break; + case V4L2_COLORSPACE_DCI_P3: + hfi_primaries = HFI_PRIMARIES_SMPTE_RP431_2; + break; + default: + break; + } + + return hfi_primaries; +} + +u32 iris_hfi_gen2_get_transfer_char(u32 characterstics) +{ + u32 hfi_characterstics = HFI_TRANSFER_RESERVED; + + switch (characterstics) { + case V4L2_XFER_FUNC_DEFAULT: + hfi_characterstics = HFI_TRANSFER_RESERVED; + break; + case V4L2_XFER_FUNC_709: + hfi_characterstics = HFI_TRANSFER_BT709; + break; + case V4L2_XFER_FUNC_SMPTE240M: + hfi_characterstics = HFI_TRANSFER_SMPTE_ST240M; + break; + case V4L2_XFER_FUNC_SRGB: + hfi_characterstics = HFI_TRANSFER_SRGB_SYCC; + break; + case V4L2_XFER_FUNC_SMPTE2084: + hfi_characterstics = HFI_TRANSFER_SMPTE_ST2084_PQ; + break; + default: + break; + } + + return hfi_characterstics; +} + +u32 iris_hfi_gen2_get_matrix_coefficients(u32 coefficients) +{ + u32 hfi_coefficients = HFI_MATRIX_COEFF_RESERVED; + + switch (coefficients) { + case V4L2_YCBCR_ENC_DEFAULT: + hfi_coefficients = HFI_MATRIX_COEFF_RESERVED; + break; + case V4L2_YCBCR_ENC_709: + hfi_coefficients = HFI_MATRIX_COEFF_BT709; + break; + case V4L2_YCBCR_ENC_XV709: + hfi_coefficients = HFI_MATRIX_COEFF_BT709; + break; + case V4L2_YCBCR_ENC_XV601: + hfi_coefficients = HFI_MATRIX_COEFF_BT470_SYS_BG_OR_BT601_625; + break; + case V4L2_YCBCR_ENC_601: + hfi_coefficients = HFI_MATRIX_COEFF_BT601_525_BT1358_525_OR_625; + break; + case V4L2_YCBCR_ENC_SMPTE240M: + hfi_coefficients = HFI_MATRIX_COEFF_SMPTE_ST240; + break; + case V4L2_YCBCR_ENC_BT2020: + hfi_coefficients = HFI_MATRIX_COEFF_BT2020_NON_CONSTANT; + break; + case V4L2_YCBCR_ENC_BT2020_CONST_LUM: + hfi_coefficients = HFI_MATRIX_COEFF_BT2020_CONSTANT; + break; + default: + break; + } + + return hfi_coefficients; +} + +u32 iris_hfi_gen2_get_color_info(u32 matrix_coeff, u32 transfer_char, u32 primaries, + u32 colour_description_present_flag, u32 full_range, + u32 video_format, u32 video_signal_type_present_flag) +{ + return (matrix_coeff & 0xFF) | + ((transfer_char << 8) & 0xFF00) | + ((primaries << 16) & 0xFF0000) | + ((colour_description_present_flag << 24) & 0x1000000) | + ((full_range << 25) & 0x2000000) | + ((video_format << 26) & 0x1C000000) | + ((video_signal_type_present_flag << 29) & 0x20000000); +} + static void iris_hfi_gen2_create_header(struct iris_hfi_header *hdr, u32 session_id, u32 header_id) { diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h index 120592322e78..bb86abd25adb 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h @@ -61,6 +61,13 @@ struct iris_hfi_packet { u32 payload[]; }; +u32 iris_hfi_gen2_get_color_primaries(u32 primaries); +u32 iris_hfi_gen2_get_transfer_char(u32 characterstics); +u32 iris_hfi_gen2_get_matrix_coefficients(u32 coefficients); +u32 iris_hfi_gen2_get_color_info(u32 matrix_coeff, u32 transfer_char, u32 primaries, + u32 colour_description_present_flag, u32 full_range, + u32 video_format, u32 video_signal_type_present_flag); + void iris_hfi_gen2_packet_sys_init(struct iris_core *core, struct iris_hfi_header *hdr); void iris_hfi_gen2_packet_image_version(struct iris_core *core, struct iris_hfi_header *hdr); void iris_hfi_gen2_packet_session_command(struct iris_inst *inst, u32 pkt_type, diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c index 35ecc66d630a..d1a26c18971f 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c @@ -158,6 +158,54 @@ static int iris_hfi_gen2_handle_session_command(struct iris_inst *inst, return 0; } +static int iris_hfi_gen2_handle_session_property(struct iris_inst *inst, + struct iris_hfi_packet *pkt) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + int ret = 0; + + if (pkt->port != HFI_PORT_BITSTREAM) + return 0; + + if (pkt->flags & HFI_FW_FLAGS_INFORMATION) + return 0; + + switch (pkt->type) { + case HFI_PROP_BITSTREAM_RESOLUTION: + inst_hfi_gen2->src_subcr_params.bitstream_resolution = pkt->payload[0]; + break; + case HFI_PROP_CROP_OFFSETS: + inst_hfi_gen2->src_subcr_params.crop_offsets[0] = pkt->payload[0]; + inst_hfi_gen2->src_subcr_params.crop_offsets[1] = pkt->payload[1]; + break; + case HFI_PROP_CODED_FRAMES: + inst_hfi_gen2->src_subcr_params.coded_frames = pkt->payload[0]; + break; + case HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT: + inst_hfi_gen2->src_subcr_params.fw_min_count = pkt->payload[0]; + break; + case HFI_PROP_PIC_ORDER_CNT_TYPE: + inst_hfi_gen2->src_subcr_params.pic_order_cnt = pkt->payload[0]; + break; + case HFI_PROP_SIGNAL_COLOR_INFO: + inst_hfi_gen2->src_subcr_params.color_info = pkt->payload[0]; + break; + case HFI_PROP_PROFILE: + inst_hfi_gen2->src_subcr_params.profile = pkt->payload[0]; + break; + case HFI_PROP_LEVEL: + inst_hfi_gen2->src_subcr_params.level = pkt->payload[0]; + break; + case HFI_PROP_QUALITY_MODE: + case HFI_PROP_STAGE: + case HFI_PROP_PIPE: + default: + break; + } + + return ret; +} + static int iris_hfi_gen2_handle_image_version_property(struct iris_core *core, struct iris_hfi_packet *pkt) { @@ -251,6 +299,8 @@ static int iris_hfi_gen2_handle_session_response(struct iris_core *core, static const struct iris_hfi_gen2_inst_hfi_range range[] = { {HFI_SESSION_ERROR_BEGIN, HFI_SESSION_ERROR_END, iris_hfi_gen2_handle_session_error}, + {HFI_PROP_BEGIN, HFI_PROP_END, + iris_hfi_gen2_handle_session_property}, {HFI_CMD_BEGIN, HFI_CMD_END, iris_hfi_gen2_handle_session_command }, }; diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h index 3893e690d314..20422bdca6d4 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_common.h +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h @@ -6,6 +6,11 @@ #ifndef _IRIS_PLATFORM_COMMON_H_ #define _IRIS_PLATFORM_COMMON_H_ +#include + +struct iris_core; +struct iris_inst; + #define IRIS_PAS_ID 9 #define HW_RESPONSE_TIMEOUT_VALUE (1000) /* milliseconds */ #define AUTOSUSPEND_DELAY_VALUE (HW_RESPONSE_TIMEOUT_VALUE + 500) /* milliseconds */ @@ -107,6 +112,8 @@ struct platform_inst_fw_cap { s64 value; u32 hfi_id; enum platform_inst_cap_flags flags; + int (*set)(struct iris_inst *inst, + enum platform_inst_fw_cap_type cap_id); }; struct iris_core_power { @@ -148,6 +155,10 @@ struct iris_platform_data { struct ubwc_config_data *ubwc_config; u32 num_vpp_pipe; u32 max_session_count; + const u32 *input_config_params; + unsigned int input_config_params_size; + const u32 *output_config_params; + unsigned int output_config_params_size; }; #endif diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c index ac9f720aff8e..6b5444b71090 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c @@ -4,6 +4,7 @@ */ #include "iris_core.h" +#include "iris_ctrls.h" #include "iris_platform_common.h" #include "iris_resources.h" #include "iris_hfi_gen1.h" @@ -40,6 +41,7 @@ static struct platform_inst_fw_cap instance_fw_cap_data_sm8250[] = { .value = PIPE_4, .hfi_id = HFI_PROPERTY_PARAM_WORK_ROUTE, .flags = CAP_FLAG_NONE, + .set = iris_set_pipe, }, { .cap_id = STAGE, @@ -49,6 +51,7 @@ static struct platform_inst_fw_cap instance_fw_cap_data_sm8250[] = { .value = STAGE_2, .hfi_id = HFI_PROPERTY_PARAM_WORK_MODE, .flags = CAP_FLAG_NONE, + .set = iris_set_stage, }, { .cap_id = DEBLOCK, @@ -58,6 +61,7 @@ static struct platform_inst_fw_cap instance_fw_cap_data_sm8250[] = { .value = 0, .hfi_id = HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER, .flags = CAP_FLAG_NONE, + .set = iris_set_u32, }, }; @@ -90,6 +94,17 @@ static struct tz_cp_config tz_cp_config_sm8250 = { .cp_nonpixel_size = 0x24800000, }; +static const u32 sm8250_vdec_input_config_param[] = { + HFI_PROPERTY_PARAM_FRAME_SIZE, + HFI_PROPERTY_CONFIG_VIDEOCORES_USAGE, + HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SELECT, + HFI_PROPERTY_PARAM_UNCOMPRESSED_PLANE_ACTUAL_CONSTRAINTS_INFO, + HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL, + HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM, + HFI_PROPERTY_PARAM_BUFFER_SIZE_ACTUAL, + HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE, +}; + struct iris_platform_data sm8250_data = { .get_instance = iris_hfi_gen1_get_instance, .init_hfi_command_ops = &iris_hfi_gen1_command_ops_init, @@ -117,4 +132,8 @@ struct iris_platform_data sm8250_data = { .hw_response_timeout = HW_RESPONSE_TIMEOUT_VALUE, .num_vpp_pipe = 4, .max_session_count = 16, + .input_config_params = + sm8250_vdec_input_config_param, + .input_config_params_size = + ARRAY_SIZE(sm8250_vdec_input_config_param), }; diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c index 42e44b454896..7be803ea867d 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c @@ -4,6 +4,7 @@ */ #include "iris_core.h" +#include "iris_ctrls.h" #include "iris_hfi_gen2.h" #include "iris_hfi_gen2_defines.h" #include "iris_platform_common.h" @@ -46,6 +47,7 @@ static struct platform_inst_fw_cap instance_fw_cap_data_sm8550[] = { .value = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, .hfi_id = HFI_PROP_PROFILE, .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + .set = iris_set_u32_enum, }, { .cap_id = LEVEL, @@ -74,6 +76,7 @@ static struct platform_inst_fw_cap instance_fw_cap_data_sm8550[] = { .value = V4L2_MPEG_VIDEO_H264_LEVEL_6_1, .hfi_id = HFI_PROP_LEVEL, .flags = CAP_FLAG_OUTPUT_PORT | CAP_FLAG_MENU, + .set = iris_set_u32_enum, }, { .cap_id = INPUT_BUF_HOST_MAX_COUNT, @@ -83,6 +86,7 @@ static struct platform_inst_fw_cap instance_fw_cap_data_sm8550[] = { .value = DEFAULT_MAX_HOST_BUF_COUNT, .hfi_id = HFI_PROP_BUFFER_HOST_MAX_COUNT, .flags = CAP_FLAG_INPUT_PORT, + .set = iris_set_u32, }, { .cap_id = STAGE, @@ -92,6 +96,7 @@ static struct platform_inst_fw_cap instance_fw_cap_data_sm8550[] = { .value = STAGE_2, .hfi_id = HFI_PROP_STAGE, .flags = CAP_FLAG_NONE, + .set = iris_set_stage, }, { .cap_id = PIPE, @@ -101,6 +106,7 @@ static struct platform_inst_fw_cap instance_fw_cap_data_sm8550[] = { .value = PIPE_4, .hfi_id = HFI_PROP_PIPE, .flags = CAP_FLAG_NONE, + .set = iris_set_pipe, }, { .cap_id = POC, @@ -142,6 +148,7 @@ static struct platform_inst_fw_cap instance_fw_cap_data_sm8550[] = { .value = 1, .hfi_id = HFI_PROP_DEC_START_FROM_RAP_FRAME, .flags = CAP_FLAG_INPUT_PORT, + .set = iris_set_u32, }, }; @@ -184,6 +191,22 @@ static struct tz_cp_config tz_cp_config_sm8550 = { .cp_nonpixel_size = 0x24800000, }; +static const u32 sm8550_vdec_input_config_params[] = { + HFI_PROP_BITSTREAM_RESOLUTION, + HFI_PROP_CROP_OFFSETS, + HFI_PROP_CODED_FRAMES, + HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT, + HFI_PROP_PIC_ORDER_CNT_TYPE, + HFI_PROP_PROFILE, + HFI_PROP_LEVEL, + HFI_PROP_SIGNAL_COLOR_INFO, +}; + +static const u32 sm8550_vdec_output_config_params[] = { + HFI_PROP_COLOR_FORMAT, + HFI_PROP_LINEAR_STRIDE_SCANLINE, +}; + struct iris_platform_data sm8550_data = { .get_instance = iris_hfi_gen2_get_instance, .init_hfi_command_ops = iris_hfi_gen2_command_ops_init, @@ -213,4 +236,12 @@ struct iris_platform_data sm8550_data = { .ubwc_config = &ubwc_config_sm8550, .num_vpp_pipe = 4, .max_session_count = 16, + .input_config_params = + sm8550_vdec_input_config_params, + .input_config_params_size = + ARRAY_SIZE(sm8550_vdec_input_config_params), + .output_config_params = + sm8550_vdec_output_config_params, + .output_config_params_size = + ARRAY_SIZE(sm8550_vdec_output_config_params), }; diff --git a/drivers/media/platform/qcom/iris/iris_utils.c b/drivers/media/platform/qcom/iris/iris_utils.c index 47e210ef4beb..7f4e08d392a1 100644 --- a/drivers/media/platform/qcom/iris/iris_utils.c +++ b/drivers/media/platform/qcom/iris/iris_utils.c @@ -8,6 +8,20 @@ #include "iris_instance.h" #include "iris_utils.h" +bool iris_res_is_less_than(u32 width, u32 height, + u32 ref_width, u32 ref_height) +{ + u32 num_mbs = NUM_MBS_PER_FRAME(height, width); + u32 max_side = max(ref_width, ref_height); + + if (num_mbs < NUM_MBS_PER_FRAME(ref_height, ref_width) && + width < max_side && + height < max_side) + return true; + + return false; +} + int iris_get_mbpf(struct iris_inst *inst) { struct v4l2_format *inp_f; @@ -20,6 +34,14 @@ int iris_get_mbpf(struct iris_inst *inst) return NUM_MBS_PER_FRAME(height, width); } +bool iris_split_mode_enabled(struct iris_inst *inst) +{ + if (inst->fmt_dst->fmt.pix_mp.pixelformat == V4L2_PIX_FMT_NV12) + return true; + + return false; +} + int iris_wait_for_session_response(struct iris_inst *inst, bool is_flush) { struct iris_core *core = inst->core; diff --git a/drivers/media/platform/qcom/iris/iris_utils.h b/drivers/media/platform/qcom/iris/iris_utils.h index a4a797fb11e2..0d46f50d3103 100644 --- a/drivers/media/platform/qcom/iris/iris_utils.h +++ b/drivers/media/platform/qcom/iris/iris_utils.h @@ -31,7 +31,10 @@ static inline enum iris_buffer_type iris_v4l2_type_to_driver(u32 type) } } +bool iris_res_is_less_than(u32 width, u32 height, + u32 ref_width, u32 ref_height); int iris_get_mbpf(struct iris_inst *inst); +bool iris_split_mode_enabled(struct iris_inst *inst); struct iris_inst *iris_get_instance(struct iris_core *core, u32 session_id); int iris_wait_for_session_response(struct iris_inst *inst, bool is_flush); diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index a2c9e9481677..ae39dd9e0dae 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -300,6 +300,12 @@ static int iris_vdec_process_streamon_input(struct iris_inst *inst) int iris_vdec_streamon_input(struct iris_inst *inst) { + int ret; + + ret = iris_set_properties(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + return iris_vdec_process_streamon_input(inst); } @@ -317,8 +323,13 @@ static int iris_vdec_process_streamon_output(struct iris_inst *inst) int iris_vdec_streamon_output(struct iris_inst *inst) { + const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; int ret; + ret = hfi_ops->session_set_config_params(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + if (ret) + return ret; + ret = iris_vdec_process_streamon_output(inst); if (ret) goto error; diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.c b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c index 2402a33723ab..7efabaebd082 100644 --- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.c +++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c @@ -6,6 +6,26 @@ #include "iris_instance.h" #include "iris_vpu_buffer.h" +u32 iris_vpu_dec_dpb_size(struct iris_inst *inst) +{ + if (iris_split_mode_enabled(inst)) + return iris_get_buffer_size(inst, BUF_DPB); + else + return 0; +} + +static inline int iris_vpu_dpb_count(struct iris_inst *inst) +{ + int count = 0; + + if (iris_split_mode_enabled(inst)) { + count = inst->fw_min_count ? + inst->fw_min_count : inst->buffers[BUF_OUTPUT].min_count; + } + + return count; +} + int iris_vpu_buf_count(struct iris_inst *inst, enum iris_buffer_type buffer_type) { switch (buffer_type) { @@ -13,6 +33,8 @@ int iris_vpu_buf_count(struct iris_inst *inst, enum iris_buffer_type buffer_type return MIN_BUFFERS; case BUF_OUTPUT: return inst->fw_min_count; + case BUF_DPB: + return iris_vpu_dpb_count(inst); default: return 0; } diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h index f0f974cebd8a..03a181c5754d 100644 --- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h +++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h @@ -10,6 +10,7 @@ struct iris_inst; #define MIN_BUFFERS 4 +u32 iris_vpu_dec_dpb_size(struct iris_inst *inst); int iris_vpu_buf_count(struct iris_inst *inst, enum iris_buffer_type buffer_type); #endif From patchwork Tue Aug 27 10:05:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 824187 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2EC541A0707; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; cv=none; b=gCLcV7QHhOeMEo8zIg32aLo4E+jVhU0HSDGM0zOmtvCfMGDER9YJkpFw7ZtZyNtl0y30kTaXE5izSzUKFXzh1UrYYgirq2/NsZ2lvy74U+Ww4XwXZf0RLpBf6oi7HbdCGUCzNjrMGfApZG25kVhOBG+W1vqJqCl7qalYx7ysJ1Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; c=relaxed/simple; bh=qFp2IWmd5gkY5X++N8b6TYA80wz7qHgX0cj0G7sqFaA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dU71qoFk6BeSvUOua+mx4xm/nCYNWy7EuObKeD6KOTt/xJDEw5YuIkUnUKiUSnXIjylo3N4XI4wgOIx0rnYdlnCvQuJcoG0ID7aRfb9oFmR09m4QHmBdbVjddLSrnDOYSWbrsOonXwVVnyNuWVYbvG057XgKByigyrDryPi4v8k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Ax3sQ276; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Ax3sQ276" Received: by smtp.kernel.org (Postfix) with ESMTPS id 0D99BC4FE12; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753196; bh=qFp2IWmd5gkY5X++N8b6TYA80wz7qHgX0cj0G7sqFaA=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=Ax3sQ276L4q9jNt8nx+0iH86SYgdQvg/N3DbeFg5K/f8YMdWeNX5QpJL5J21hCaoe HySTj0DYgyK3+NhA9UQAQO1zHjkFsODqska2CBeYYPDfWwY0+L/R5r7+iDVXaC8fE7 A30H0DNxx6KfcIHjoy+YL7sftX4fYKJhCMPmNDjhvEJ7TAeQGzdwSQBHBHf6beE/+f 3jkJA16mH5R+jXpSq4aQ0ua17UzpGwobITsqjTIpnGoAwY01m9vtfTen/Bm0VKnN0w jRUMk2FQrDWaA7xDUCM8+MQ9yA+fj8YqQ+XQh7XeABs/upG7EQEdv5ckDvqg1H27Ul 0afR3VblqzurQ== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04613C54735; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:45 +0530 Subject: [PATCH v3 20/29] media: iris: subscribe parameters and properties to firmware for hfi_gen2 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-20-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal , Vedang Nagar X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=11506; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=/yH5GYpTmFH04bcjTHhzyHyIgbmxEkMWOptBtTwxBPE=; b=bRQaO2j465ExU+8ZUQHwORBRx8la1gamApl2CY2FyYpeGmEII5vOXtvqWfs6VPcfqj63E6y2P t9WFGWtZRvRDaEZXqClEjHcrIKDXwO88kiZFmwRcmjjK19l83ozeCp6 X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Vedang Nagar For hfi_gen2, subscribe for different bitstream parameters to firmware to get notified for change in any of the subscribed parameters. Signed-off-by: Vedang Nagar Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/iris_hfi_gen2.h | 6 + .../platform/qcom/iris/iris_hfi_gen2_command.c | 179 +++++++++++++++++++++ .../platform/qcom/iris/iris_hfi_gen2_defines.h | 9 ++ .../platform/qcom/iris/iris_platform_common.h | 4 + .../platform/qcom/iris/iris_platform_sm8550.c | 13 ++ 5 files changed, 211 insertions(+) diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2.h index 8170c1fef569..5fbbab844025 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2.h @@ -18,12 +18,18 @@ struct iris_core; * * @inst: pointer to iris_instance structure * @packet: HFI packet + * @ipsc_properties_set: boolean to set ipsc properties to fw + * @opsc_properties_set: boolean to set opsc properties to fw * @src_subcr_params: subscription params to fw on input port + * @dst_subcr_params: subscription params to fw on output port */ struct iris_inst_hfi_gen2 { struct iris_inst inst; struct iris_hfi_header *packet; + bool ipsc_properties_set; + bool opsc_properties_set; struct hfi_subscription_params src_subcr_params; + struct hfi_subscription_params dst_subcr_params; }; void iris_hfi_gen2_command_ops_init(struct iris_core *core); diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c index e50f00021f6d..791b535a3584 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c @@ -472,6 +472,9 @@ static int iris_hfi_gen2_session_open(struct iris_inst *inst) if (inst->state != IRIS_INST_DEINIT) return -EALREADY; + inst_hfi_gen2->ipsc_properties_set = false; + inst_hfi_gen2->opsc_properties_set = false; + inst_hfi_gen2->packet = kzalloc(4096, GFP_KERNEL); if (!inst_hfi_gen2->packet) return -ENOMEM; @@ -536,9 +539,185 @@ static int iris_hfi_gen2_session_close(struct iris_inst *inst) return ret; } +static int iris_hfi_gen2_session_subscribe_mode(struct iris_inst *inst, + u32 cmd, u32 plane, u32 payload_type, + void *payload, u32 payload_size) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + + iris_hfi_gen2_packet_session_command(inst, + cmd, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED), + iris_hfi_gen2_get_port(plane), + inst->session_id, + payload_type, + payload, + payload_size); + + return iris_hfi_queue_cmd_write(inst->core, inst_hfi_gen2->packet, + inst_hfi_gen2->packet->size); +} + +static int iris_hfi_gen2_subscribe_change_param(struct iris_inst *inst, u32 plane) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + struct hfi_subscription_params subsc_params; + u32 prop_type, payload_size, payload_type; + struct iris_core *core = inst->core; + const u32 *change_param = NULL; + u32 change_param_size = 0; + u32 payload[32] = {0}; + u32 hfi_port = 0; + int ret; + u32 i; + + if ((V4L2_TYPE_IS_OUTPUT(plane) && inst_hfi_gen2->ipsc_properties_set) || + (V4L2_TYPE_IS_CAPTURE(plane) && inst_hfi_gen2->opsc_properties_set)) { + dev_err(core->dev, "invalid plane\n"); + return 0; + } + + change_param = core->iris_platform_data->input_config_params; + change_param_size = core->iris_platform_data->input_config_params_size; + + if (!change_param || !change_param_size) + return -EINVAL; + + payload[0] = HFI_MODE_PORT_SETTINGS_CHANGE; + + for (i = 0; i < change_param_size; i++) + payload[i + 1] = change_param[i]; + + ret = iris_hfi_gen2_session_subscribe_mode(inst, + HFI_CMD_SUBSCRIBE_MODE, + plane, + HFI_PAYLOAD_U32_ARRAY, + &payload[0], + ((change_param_size + 1) * sizeof(u32))); + if (ret) + return ret; + + if (V4L2_TYPE_IS_OUTPUT(plane)) { + inst_hfi_gen2->ipsc_properties_set = true; + } else { + hfi_port = iris_hfi_gen2_get_port(V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + memcpy(&inst_hfi_gen2->dst_subcr_params, + &inst_hfi_gen2->src_subcr_params, + sizeof(inst_hfi_gen2->src_subcr_params)); + subsc_params = inst_hfi_gen2->dst_subcr_params; + for (i = 0; i < change_param_size; i++) { + payload[0] = 0; + payload[1] = 0; + payload_size = 0; + payload_type = 0; + prop_type = change_param[i]; + switch (prop_type) { + case HFI_PROP_BITSTREAM_RESOLUTION: + payload[0] = subsc_params.bitstream_resolution; + payload_size = sizeof(u32); + payload_type = HFI_PAYLOAD_U32; + break; + case HFI_PROP_CROP_OFFSETS: + payload[0] = subsc_params.crop_offsets[0]; + payload[1] = subsc_params.crop_offsets[1]; + payload_size = sizeof(u64); + payload_type = HFI_PAYLOAD_64_PACKED; + break; + case HFI_PROP_CODED_FRAMES: + payload[0] = subsc_params.coded_frames; + payload_size = sizeof(u32); + payload_type = HFI_PAYLOAD_U32; + break; + case HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT: + payload[0] = subsc_params.fw_min_count; + payload_size = sizeof(u32); + payload_type = HFI_PAYLOAD_U32; + break; + case HFI_PROP_PIC_ORDER_CNT_TYPE: + payload[0] = subsc_params.pic_order_cnt; + payload_size = sizeof(u32); + payload_type = HFI_PAYLOAD_U32; + break; + case HFI_PROP_SIGNAL_COLOR_INFO: + payload[0] = subsc_params.color_info; + payload_size = sizeof(u32); + payload_type = HFI_PAYLOAD_U32; + break; + case HFI_PROP_PROFILE: + payload[0] = subsc_params.profile; + payload_size = sizeof(u32); + payload_type = HFI_PAYLOAD_U32; + break; + case HFI_PROP_LEVEL: + payload[0] = subsc_params.level; + payload_size = sizeof(u32); + payload_type = HFI_PAYLOAD_U32; + break; + default: + prop_type = 0; + ret = -EINVAL; + break; + } + if (prop_type) { + ret = iris_hfi_gen2_session_set_property(inst, + prop_type, + HFI_HOST_FLAGS_NONE, + hfi_port, + payload_type, + &payload, + payload_size); + if (ret) + return ret; + } + } + inst_hfi_gen2->opsc_properties_set = true; + } + + return ret; +} + +static int iris_hfi_gen2_subscribe_property(struct iris_inst *inst, u32 plane) +{ + struct iris_core *core = inst->core; + const u32 *subcribe_prop = NULL; + u32 subscribe_prop_size = 0; + u32 payload[32] = {0}; + u32 i; + + payload[0] = HFI_MODE_PROPERTY; + + if (V4L2_TYPE_IS_OUTPUT(plane)) { + subscribe_prop_size = core->iris_platform_data->dec_input_prop_size; + subcribe_prop = core->iris_platform_data->dec_input_prop; + } else { + subscribe_prop_size = core->iris_platform_data->dec_output_prop_size; + subcribe_prop = core->iris_platform_data->dec_output_prop; + } + + for (i = 0; i < subscribe_prop_size; i++) + payload[i + 1] = subcribe_prop[i]; + + return iris_hfi_gen2_session_subscribe_mode(inst, + HFI_CMD_SUBSCRIBE_MODE, + plane, + HFI_PAYLOAD_U32_ARRAY, + &payload[0], + (subscribe_prop_size + 1) * sizeof(u32)); +} + static int iris_hfi_gen2_session_start(struct iris_inst *inst, u32 plane) { struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + int ret = 0; + + ret = iris_hfi_gen2_subscribe_change_param(inst, plane); + if (ret) + return ret; + + ret = iris_hfi_gen2_subscribe_property(inst, plane); + if (ret) + return ret; iris_hfi_gen2_packet_session_command(inst, HFI_CMD_START, diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h index b5b232d31ad8..92c44841c67d 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h @@ -17,6 +17,7 @@ #define HFI_CMD_CLOSE 0x01000004 #define HFI_CMD_START 0x01000005 #define HFI_CMD_STOP 0x01000006 +#define HFI_CMD_SUBSCRIBE_MODE 0x0100000B #define HFI_CMD_END 0x01FFFFFF #define HFI_BITMASK_FRAME_MBS_ONLY_FLAG 0x00000001 @@ -42,13 +43,16 @@ #define HFI_PROP_PIPE 0x0300010b #define HFI_PROP_LUMA_CHROMA_BIT_DEPTH 0x0300010f #define HFI_PROP_CODED_FRAMES 0x03000120 +#define HFI_PROP_CABAC_SESSION 0x03000121 #define HFI_PROP_BUFFER_HOST_MAX_COUNT 0x03000123 #define HFI_PROP_BUFFER_FW_MIN_OUTPUT_COUNT 0x03000124 #define HFI_PROP_PIC_ORDER_CNT_TYPE 0x03000128 #define HFI_PROP_QUALITY_MODE 0x03000148 #define HFI_PROP_SIGNAL_COLOR_INFO 0x03000155 +#define HFI_PROP_PICTURE_TYPE 0x03000162 #define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168 #define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169 +#define HFI_PROP_NO_OUTPUT 0x0300016a #define HFI_PROP_END 0x03FFFFFF #define HFI_SESSION_ERROR_BEGIN 0x04000000 @@ -64,6 +68,11 @@ #define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001 #define HFI_SYSTEM_ERROR_END 0x05FFFFFF +enum hfi_property_mode_type { + HFI_MODE_PORT_SETTINGS_CHANGE = 0x00000001, + HFI_MODE_PROPERTY = 0x00000002, +}; + enum hfi_color_format { HFI_COLOR_FMT_OPAQUE = 0, HFI_COLOR_FMT_NV12 = 1, diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h index 20422bdca6d4..5a01391845c6 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_common.h +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h @@ -159,6 +159,10 @@ struct iris_platform_data { unsigned int input_config_params_size; const u32 *output_config_params; unsigned int output_config_params_size; + const u32 *dec_input_prop; + unsigned int dec_input_prop_size; + const u32 *dec_output_prop; + unsigned int dec_output_prop_size; }; #endif diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c index 7be803ea867d..fccee5309d81 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c @@ -207,6 +207,15 @@ static const u32 sm8550_vdec_output_config_params[] = { HFI_PROP_LINEAR_STRIDE_SCANLINE, }; +static const u32 sm8550_vdec_subscribe_input_properties[] = { + HFI_PROP_NO_OUTPUT, +}; + +static const u32 sm8550_vdec_subscribe_output_properties[] = { + HFI_PROP_PICTURE_TYPE, + HFI_PROP_CABAC_SESSION, +}; + struct iris_platform_data sm8550_data = { .get_instance = iris_hfi_gen2_get_instance, .init_hfi_command_ops = iris_hfi_gen2_command_ops_init, @@ -244,4 +253,8 @@ struct iris_platform_data sm8550_data = { sm8550_vdec_output_config_params, .output_config_params_size = ARRAY_SIZE(sm8550_vdec_output_config_params), + .dec_input_prop = sm8550_vdec_subscribe_input_properties, + .dec_input_prop_size = ARRAY_SIZE(sm8550_vdec_subscribe_input_properties), + .dec_output_prop = sm8550_vdec_subscribe_output_properties, + .dec_output_prop_size = ARRAY_SIZE(sm8550_vdec_subscribe_output_properties), }; From patchwork Tue Aug 27 10:05:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 822720 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5C4101A0730; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; cv=none; b=oq/c8w0BT5LPOHMJjuhBChYFlWCwI/c/Of03VxnlJgeO+D2o4y3WT+u1AjUGDf+Tv3CxWoLbpvEWS3ZuY/GL4B68Ty5tAV9m4PqcLrax8c778xZ9v+7YJFt4Ci+TeDKgEV1fsbQyp3NsAHW3I+bHec3A8yn5gUz1fNOMif/3W4Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; c=relaxed/simple; bh=n8mIfDDfCVCKT/qRJY/PHdFn+vxZRxRzlDKgMgQKZUM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=eUep8w0gx2bYVLs2vARLtAUCUy9CzIJgGKmUH2tbqUbQsvjXLC1SkMqQ8YwIJ9OiXFawXp93F/+jCiBjtXtlvamRW8dMG3PSPV0w8QBQdwCTZQhL74pCpwe6si+8XYGpjdEYJgnoRr3EqY/chjNr/E/TFMCK938+0Z70ZwTxcXA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UEliKjqz; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UEliKjqz" Received: by smtp.kernel.org (Postfix) with ESMTPS id 39BA5C4FE8F; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753196; bh=n8mIfDDfCVCKT/qRJY/PHdFn+vxZRxRzlDKgMgQKZUM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=UEliKjqzsC71buHoDMnLfjEtctbJaonf7SoxzmZv0Sg7oPvBl7N/lVdgPPwsZ6MT/ 9gscahFEuzAEP7SagXMetDZfFSrWiP0L6visQHGgB1P7/B1l9oWxBkqGuyLfab9xvW R5/7c8oDYFQTJ9xAwtXNGaSazK1D+rwtc6oPkG+etfFoUvywBtlaFi7SUNBAyYThxd ukT9jYd8I0Uxj/Wz9m+tjwA5fDBQnp3jVo+EBa0pWaQYbqwatwl6nvwMDTCbjlaO3j Ak2ogOhl24CHM0nRKNEkx5Zj8FcjtIdWz9ZdtxYcuoNZqjq1BOv/PukhLV94V5Cj// DMOHPTZa+Y0xg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24252C54736; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:46 +0530 Subject: [PATCH v3 21/29] media: iris: allocate, initialize and queue internal buffers Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-21-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=50983; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=lxXt5Hhd5blgzOczVEZFGbf948OHw/aPfN4kIOpUbek=; b=EwYO1vhlh93THc4E6cRHTzjqvSuIIQJFS4/vUe1Ytt+W3E8MRfBArRRM2gRVpxskaOyl4fc1G 2n5g4M73TNLD0lCHzrPIuqRTrBKPWtgN5Gw4wgv1qhJ56WZfN7O0U7h X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal Implement functions for creating, queueing, releasing and destroying buffers for internal usage. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/iris_buffer.c | 224 +++++++++++++++++ drivers/media/platform/qcom/iris/iris_buffer.h | 7 + drivers/media/platform/qcom/iris/iris_hfi_common.h | 4 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 128 +++++++++- .../platform/qcom/iris/iris_hfi_gen1_defines.h | 37 +++ .../platform/qcom/iris/iris_hfi_gen1_response.c | 4 + .../platform/qcom/iris/iris_hfi_gen2_command.c | 138 +++++++++++ .../platform/qcom/iris/iris_hfi_gen2_defines.h | 9 + .../platform/qcom/iris/iris_hfi_gen2_packet.h | 41 ++++ .../platform/qcom/iris/iris_hfi_gen2_response.c | 148 ++++++++++- .../platform/qcom/iris/iris_platform_common.h | 5 + .../platform/qcom/iris/iris_platform_sm8250.c | 14 ++ .../platform/qcom/iris/iris_platform_sm8550.c | 22 ++ drivers/media/platform/qcom/iris/iris_vdec.c | 38 ++- drivers/media/platform/qcom/iris/iris_vidc.c | 11 + drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 272 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 77 +++++- 17 files changed, 1174 insertions(+), 5 deletions(-) diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c b/drivers/media/platform/qcom/iris/iris_buffer.c index b90b912d73b6..57f3ffd8596a 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/iris/iris_buffer.c @@ -7,6 +7,7 @@ #include "iris_buffer.h" #include "iris_instance.h" +#include "iris_vpu_buffer.h" #define PIXELS_4K 4096 #define MAX_WIDTH 4096 @@ -215,6 +216,229 @@ int iris_get_buffer_size(struct iris_inst *inst, } } +static void iris_fill_internal_buf_info(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + struct iris_core *core = inst->core; + struct iris_buffers *buffers; + + buffers = &inst->buffers[buffer_type]; + + buffers->size = iris_vpu_buf_size(inst, buffer_type); + buffers->min_count = iris_vpu_buf_count(inst, buffer_type); + + dev_dbg(core->dev, "buffer type %d count %d size %d", + buffer_type, buffers->min_count, buffers->size); +} + +void iris_get_internal_buffers(struct iris_inst *inst, u32 plane) +{ + const struct iris_platform_data *platform_data; + const u32 *internal_buf_type; + u32 internal_buffer_count; + u32 i = 0; + + platform_data = inst->core->iris_platform_data; + + if (V4L2_TYPE_IS_OUTPUT(plane)) { + internal_buf_type = platform_data->dec_ip_int_buf_tbl; + internal_buffer_count = platform_data->dec_ip_int_buf_tbl_size; + for (i = 0; i < internal_buffer_count; i++) + iris_fill_internal_buf_info(inst, internal_buf_type[i]); + } else { + internal_buf_type = platform_data->dec_op_int_buf_tbl; + internal_buffer_count = platform_data->dec_op_int_buf_tbl_size; + for (i = 0; i < internal_buffer_count; i++) + iris_fill_internal_buf_info(inst, internal_buf_type[i]); + } +} + +static int iris_create_internal_buffer(struct iris_inst *inst, + enum iris_buffer_type buffer_type, u32 index) +{ + struct iris_core *core = inst->core; + struct iris_buffers *buffers; + struct iris_buffer *buffer; + + buffers = &inst->buffers[buffer_type]; + + if (!buffers->size) + return 0; + + buffer = kzalloc(sizeof(*buffer), GFP_KERNEL); + if (!buffer) + return -ENOMEM; + + INIT_LIST_HEAD(&buffer->list); + buffer->type = buffer_type; + buffer->index = index; + buffer->buffer_size = buffers->size; + buffer->dma_attrs = DMA_ATTR_WRITE_COMBINE | DMA_ATTR_NO_KERNEL_MAPPING; + list_add_tail(&buffer->list, &buffers->list); + + buffer->kvaddr = dma_alloc_attrs(core->dev, buffer->buffer_size, + &buffer->device_addr, GFP_KERNEL, buffer->dma_attrs); + if (!buffer->kvaddr) + return -ENOMEM; + + return 0; +} + +int iris_create_internal_buffers(struct iris_inst *inst, u32 plane) +{ + const struct iris_platform_data *platform_data; + struct iris_buffers *buffers; + const u32 *internal_buf_type; + u32 internal_buffer_count; + u32 i = 0, j = 0; + int ret = 0; + + platform_data = inst->core->iris_platform_data; + + if (V4L2_TYPE_IS_OUTPUT(plane)) { + internal_buf_type = platform_data->dec_ip_int_buf_tbl; + internal_buffer_count = platform_data->dec_ip_int_buf_tbl_size; + } else { + internal_buf_type = platform_data->dec_op_int_buf_tbl; + internal_buffer_count = platform_data->dec_op_int_buf_tbl_size; + } + + for (i = 0; i < internal_buffer_count; i++) { + buffers = &inst->buffers[internal_buf_type[i]]; + for (j = 0; j < buffers->min_count; j++) { + ret = iris_create_internal_buffer(inst, internal_buf_type[i], j); + if (ret) + return ret; + } + } + + return ret; +} + +int iris_queue_buffer(struct iris_inst *inst, struct iris_buffer *buf) +{ + const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + int ret; + + ret = hfi_ops->session_queue_buf(inst, buf); + if (ret) + return ret; + + buf->attr &= ~BUF_ATTR_DEFERRED; + buf->attr |= BUF_ATTR_QUEUED; + + return ret; +} + +int iris_queue_internal_buffers(struct iris_inst *inst, u32 plane) +{ + const struct iris_platform_data *platform_data; + struct iris_buffer *buffer, *next; + struct iris_buffers *buffers; + const u32 *internal_buf_type; + u32 internal_buffer_count; + int ret = 0; + u32 i = 0; + + platform_data = inst->core->iris_platform_data; + + if (V4L2_TYPE_IS_OUTPUT(plane)) { + internal_buf_type = platform_data->dec_ip_int_buf_tbl; + internal_buffer_count = platform_data->dec_ip_int_buf_tbl_size; + } else { + internal_buf_type = platform_data->dec_op_int_buf_tbl; + internal_buffer_count = platform_data->dec_op_int_buf_tbl_size; + } + + for (i = 0; i < internal_buffer_count; i++) { + buffers = &inst->buffers[internal_buf_type[i]]; + list_for_each_entry_safe(buffer, next, &buffers->list, list) { + if (buffer->attr & BUF_ATTR_PENDING_RELEASE) + continue; + if (buffer->attr & BUF_ATTR_QUEUED) + continue; + ret = iris_queue_buffer(inst, buffer); + if (ret) + return ret; + } + } + + return ret; +} + +int iris_destroy_internal_buffer(struct iris_inst *inst, struct iris_buffer *buffer) +{ + struct iris_core *core = inst->core; + + list_del(&buffer->list); + dma_free_attrs(core->dev, buffer->buffer_size, buffer->kvaddr, + buffer->device_addr, buffer->dma_attrs); + kfree(buffer); + + return 0; +} + +int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane) +{ + const struct iris_platform_data *platform_data; + const u32 *internal_buf_type = NULL; + struct iris_buffer *buf, *next; + struct iris_buffers *buffers; + int ret = 0; + u32 i, len = 0; + + platform_data = inst->core->iris_platform_data; + + if (V4L2_TYPE_IS_OUTPUT(plane)) { + internal_buf_type = platform_data->dec_ip_int_buf_tbl; + len = platform_data->dec_ip_int_buf_tbl_size; + } else { + internal_buf_type = platform_data->dec_op_int_buf_tbl; + len = platform_data->dec_op_int_buf_tbl_size; + } + + for (i = 0; i < len; i++) { + buffers = &inst->buffers[internal_buf_type[i]]; + list_for_each_entry_safe(buf, next, &buffers->list, list) { + ret = iris_destroy_internal_buffer(inst, buf); + if (ret) + return ret; + } + } + + return ret; +} + +int iris_alloc_and_queue_persist_bufs(struct iris_inst *inst) +{ + struct iris_buffer *buffer, *next; + struct iris_buffers *buffers; + int ret = 0; + int i = 0; + + iris_fill_internal_buf_info(inst, BUF_PERSIST); + + buffers = &inst->buffers[BUF_PERSIST]; + + for (i = 0; i < buffers->min_count; i++) { + ret = iris_create_internal_buffer(inst, BUF_PERSIST, i); + if (ret) + return ret; + } + + list_for_each_entry_safe(buffer, next, &buffers->list, list) { + if (buffer->attr & BUF_ATTR_PENDING_RELEASE) + continue; + if (buffer->attr & BUF_ATTR_QUEUED) + continue; + ret = iris_queue_buffer(inst, buffer); + if (ret) + return ret; + } + + return ret; +} + void iris_vb2_queue_error(struct iris_inst *inst) { struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h index 98844e89e0e3..41e090e9f0b0 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.h +++ b/drivers/media/platform/qcom/iris/iris_buffer.h @@ -102,6 +102,13 @@ struct iris_buffers { }; int iris_get_buffer_size(struct iris_inst *inst, enum iris_buffer_type buffer_type); +void iris_get_internal_buffers(struct iris_inst *inst, u32 plane); +int iris_create_internal_buffers(struct iris_inst *inst, u32 plane); +int iris_queue_internal_buffers(struct iris_inst *inst, u32 plane); +int iris_destroy_internal_buffer(struct iris_inst *inst, struct iris_buffer *buffer); +int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane); +int iris_alloc_and_queue_persist_bufs(struct iris_inst *inst); +int iris_queue_buffer(struct iris_inst *inst, struct iris_buffer *buf); void iris_vb2_queue_error(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.h b/drivers/media/platform/qcom/iris/iris_hfi_common.h index fa409a9b3f04..9a6bb72e6134 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_common.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_common.h @@ -9,6 +9,8 @@ #include #include +#include "iris_buffer.h" + struct iris_inst; struct iris_core; @@ -114,6 +116,8 @@ struct iris_hfi_command_ops { void *payload, u32 payload_size); int (*session_open)(struct iris_inst *inst); int (*session_start)(struct iris_inst *inst, u32 plane); + int (*session_queue_buf)(struct iris_inst *inst, struct iris_buffer *buffer); + int (*session_release_buf)(struct iris_inst *inst, struct iris_buffer *buffer); int (*session_stop)(struct iris_inst *inst, u32 plane); int (*session_close)(struct iris_inst *inst); }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c index 65f561ec76ab..be45a38166eb 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c @@ -8,6 +8,24 @@ #include "iris_instance.h" #include "iris_vpu_buffer.h" +static u32 iris_hfi_gen1_buf_type_from_driver(enum iris_buffer_type buffer_type) +{ + switch (buffer_type) { + case BUF_INPUT: + return HFI_BUFFER_INPUT; + case BUF_OUTPUT: + return HFI_BUFFER_OUTPUT; + case BUF_PERSIST: + return HFI_BUFFER_INTERNAL_PERSIST_1; + case BUF_BIN: + return HFI_BUFFER_INTERNAL_SCRATCH; + case BUF_SCRATCH_1: + return HFI_BUFFER_INTERNAL_SCRATCH_1; + default: + return -EINVAL; + } +} + static int iris_hfi_gen1_sys_init(struct iris_core *core) { struct hfi_sys_init_pkt sys_init_pkt; @@ -183,6 +201,112 @@ static int iris_hfi_gen1_session_stop(struct iris_inst *inst, u32 plane) return ret; } +static int iris_hfi_gen1_queue_internal_buffer(struct iris_inst *inst, struct iris_buffer *buf) +{ + struct hfi_session_set_buffers_pkt *int_pkt; + int buffer_type, i; + u32 packet_size; + int ret; + + packet_size = struct_size(int_pkt, buffer_info, 1); + int_pkt = kzalloc(packet_size, GFP_KERNEL); + if (!int_pkt) + return -ENOMEM; + + int_pkt->shdr.hdr.pkt_type = HFI_CMD_SESSION_SET_BUFFERS; + int_pkt->shdr.session_id = inst->session_id; + int_pkt->buffer_size = buf->buffer_size; + int_pkt->min_buffer_size = buf->buffer_size; + int_pkt->num_buffers = 1; + int_pkt->extradata_size = 0; + int_pkt->shdr.hdr.size = packet_size; + for (i = 0; i < int_pkt->num_buffers; i++) + int_pkt->buffer_info[i] = buf->device_addr; + buffer_type = iris_hfi_gen1_buf_type_from_driver(buf->type); + if (buffer_type == -EINVAL) { + ret = -EINVAL; + goto exit; + } + + int_pkt->buffer_type = buffer_type; + ret = iris_hfi_queue_cmd_write(inst->core, int_pkt, int_pkt->shdr.hdr.size); + +exit: + kfree(int_pkt); + + return ret; +} + +static int iris_hfi_gen1_session_queue_buffer(struct iris_inst *inst, struct iris_buffer *buf) +{ + switch (buf->type) { + case BUF_PERSIST: + case BUF_BIN: + case BUF_SCRATCH_1: + return iris_hfi_gen1_queue_internal_buffer(inst, buf); + default: + return -EINVAL; + } +} + +static int iris_hfi_gen1_session_unset_buffers(struct iris_inst *inst, struct iris_buffer *buf) +{ + struct hfi_session_release_buffer_pkt *pkt; + unsigned int buffer_type = 0, i; + u32 packet_size; + int ret; + + buffer_type = iris_hfi_gen1_buf_type_from_driver(buf->type); + if (buffer_type == -EINVAL) + return -EINVAL; + + if (buffer_type == HFI_BUFFER_INPUT) + return 0; + + packet_size = sizeof(*pkt) + sizeof(struct hfi_buffer_info); + pkt = kzalloc(packet_size, GFP_KERNEL); + if (!pkt) + return -ENOMEM; + + pkt->shdr.hdr.pkt_type = HFI_CMD_SESSION_RELEASE_BUFFERS; + pkt->shdr.session_id = inst->session_id; + pkt->buffer_size = buf->buffer_size; + pkt->num_buffers = 1; + + if (buffer_type == HFI_BUFFER_OUTPUT || + buffer_type == HFI_BUFFER_OUTPUT2) { + struct hfi_buffer_info *bi; + + bi = (struct hfi_buffer_info *)pkt->buffer_info; + for (i = 0; i < pkt->num_buffers; i++) { + bi->buffer_addr = buf->device_addr; + bi->extradata_addr = 0; + } + pkt->shdr.hdr.size = packet_size; + } else { + for (i = 0; i < pkt->num_buffers; i++) + pkt->buffer_info[i] = buf->device_addr; + pkt->extradata_size = 0; + pkt->shdr.hdr.size = + sizeof(struct hfi_session_set_buffers_pkt) + + ((pkt->num_buffers) * sizeof(u32)); + } + + pkt->response_req = true; + pkt->buffer_type = buffer_type; + + ret = iris_hfi_queue_cmd_write(inst->core, pkt, pkt->shdr.hdr.size); + if (ret) + goto exit; + + ret = iris_wait_for_session_response(inst, false); + +exit: + kfree(pkt); + + return ret; +} + static int iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *packet, struct iris_inst *inst, u32 ptype, void *pdata) @@ -498,7 +622,7 @@ static int iris_hfi_gen1_set_bufsize(struct iris_inst *inst) if (iris_split_mode_enabled(inst)) { bufsz.type = HFI_BUFFER_OUTPUT; - bufsz.size = iris_vpu_dec_dpb_size(inst); + bufsz.size = iris_vpu_buf_size(inst, BUF_DPB); ret = hfi_gen1_set_property(inst, ptype, &bufsz, sizeof(bufsz)); if (ret) @@ -607,6 +731,8 @@ static const struct iris_hfi_command_ops iris_hfi_gen1_command_ops = { .session_set_config_params = iris_hfi_gen1_session_set_config_params, .session_set_property = iris_hfi_gen1_session_set_property, .session_start = iris_hfi_gen1_session_start, + .session_queue_buf = iris_hfi_gen1_session_queue_buffer, + .session_release_buf = iris_hfi_gen1_session_unset_buffers, .session_stop = iris_hfi_gen1_session_stop, .session_close = iris_hfi_gen1_session_close, }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h index 8b48a082bcd3..5527233b03e5 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h @@ -24,11 +24,13 @@ #define HFI_CMD_SYS_SESSION_END 0x10008 #define HFI_CMD_SESSION_SET_PROPERTY 0x11001 +#define HFI_CMD_SESSION_SET_BUFFERS 0x11002 #define HFI_CMD_SESSION_LOAD_RESOURCES 0x211001 #define HFI_CMD_SESSION_START 0x211002 #define HFI_CMD_SESSION_STOP 0x211003 #define HFI_CMD_SESSION_FLUSH 0x211008 +#define HFI_CMD_SESSION_RELEASE_BUFFERS 0x21100b #define HFI_CMD_SESSION_RELEASE_RESOURCES 0x21100c #define HFI_ERR_SESSION_UNSUPPORTED_SETTING 0x1008 @@ -53,6 +55,9 @@ #define HFI_BUFFER_INPUT 0x1 #define HFI_BUFFER_OUTPUT 0x2 #define HFI_BUFFER_OUTPUT2 0x3 +#define HFI_BUFFER_INTERNAL_PERSIST_1 0x5 +#define HFI_BUFFER_INTERNAL_SCRATCH 0x6 +#define HFI_BUFFER_INTERNAL_SCRATCH_1 0x7 #define HFI_PROPERTY_SYS_CODEC_POWER_PLANE_CTRL 0x5 #define HFI_PROPERTY_SYS_IMAGE_VERSION 0x6 @@ -80,6 +85,7 @@ #define HFI_MSG_SESSION_STOP 0x221003 #define HFI_MSG_SESSION_FLUSH 0x221006 #define HFI_MSG_SESSION_RELEASE_RESOURCES 0x22100a +#define HFI_MSG_SESSION_RELEASE_BUFFERS 0x22100c struct hfi_pkt_hdr { u32 size; @@ -128,11 +134,36 @@ struct hfi_sys_pc_prep_pkt { struct hfi_pkt_hdr hdr; }; +struct hfi_session_set_buffers_pkt { + struct hfi_session_hdr_pkt shdr; + u32 buffer_type; + u32 buffer_size; + u32 extradata_size; + u32 min_buffer_size; + u32 num_buffers; + u32 buffer_info[]; +}; + struct hfi_session_flush_pkt { struct hfi_session_hdr_pkt shdr; u32 flush_type; }; +struct hfi_session_release_buffer_pkt { + struct hfi_session_hdr_pkt shdr; + u32 buffer_type; + u32 buffer_size; + u32 extradata_size; + u32 response_req; + u32 num_buffers; + u32 buffer_info[]; +}; + +struct hfi_buffer_info { + u32 buffer_addr; + u32 extradata_addr; +}; + struct hfi_msg_event_notify_pkt { struct hfi_session_hdr_pkt shdr; u32 event_id; @@ -227,6 +258,12 @@ struct hfi_multi_stream { u32 enable; }; +struct hfi_msg_session_release_buffers_done_pkt { + struct hfi_msg_session_hdr_pkt shdr; + u32 num_buffers; + u32 buffer_info[]; +}; + struct hfi_msg_sys_debug_pkt { struct hfi_pkt_hdr hdr; u32 msg_type; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c index b42eba10e074..af1f59c43b92 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c @@ -169,6 +169,10 @@ static const struct iris_hfi_gen1_response_pkt_info pkt_infos[] = { .pkt = HFI_MSG_SESSION_RELEASE_RESOURCES, .pkt_sz = sizeof(struct hfi_msg_session_hdr_pkt), }, + { + .pkt = HFI_MSG_SESSION_RELEASE_BUFFERS, + .pkt_sz = sizeof(struct hfi_msg_session_release_buffers_done_pkt), + }, }; static void iris_hfi_gen1_handle_response(struct iris_core *core, void *response) diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c index 791b535a3584..ab1021c3b562 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c @@ -106,6 +106,32 @@ static u32 iris_hfi_gen2_get_port(u32 plane) return hfi_port; } +static u32 iris_hfi_gen2_get_port_from_buf_type(enum iris_buffer_type buffer_type) +{ + u32 hfi_port = HFI_PORT_NONE; + + switch (buffer_type) { + case BUF_INPUT: + case BUF_BIN: + case BUF_COMV: + case BUF_NON_COMV: + case BUF_LINE: + hfi_port = HFI_PORT_BITSTREAM; + break; + case BUF_OUTPUT: + case BUF_DPB: + hfi_port = HFI_PORT_RAW; + break; + case BUF_PERSIST: + hfi_port = HFI_PORT_NONE; + break; + default: + break; + } + + return hfi_port; +} + static int iris_hfi_gen2_session_set_property(struct iris_inst *inst, u32 packet_type, u32 flag, u32 plane, u32 payload_type, void *payload, u32 payload_size) @@ -759,6 +785,116 @@ static int iris_hfi_gen2_session_stop(struct iris_inst *inst, u32 plane) return iris_wait_for_session_response(inst, false); } +static u32 iris_hfi_gen2_buf_type_from_driver(enum iris_buffer_type buffer_type) +{ + switch (buffer_type) { + case BUF_INPUT: + return HFI_BUFFER_BITSTREAM; + case BUF_OUTPUT: + return HFI_BUFFER_RAW; + case BUF_BIN: + return HFI_BUFFER_BIN; + case BUF_COMV: + return HFI_BUFFER_COMV; + case BUF_NON_COMV: + return HFI_BUFFER_NON_COMV; + case BUF_LINE: + return HFI_BUFFER_LINE; + case BUF_DPB: + return HFI_BUFFER_DPB; + case BUF_PERSIST: + return HFI_BUFFER_PERSIST; + default: + return 0; + } +} + +static int iris_set_num_comv(struct iris_inst *inst) +{ + struct iris_core *core = inst->core; + u32 num_comv; + + num_comv = inst->driver_cap[NUM_COMV].value; + + return core->hfi_ops->session_set_property(inst, + HFI_PROP_COMV_BUFFER_COUNT, + HFI_HOST_FLAGS_NONE, + HFI_PORT_BITSTREAM, + HFI_PAYLOAD_U32, + &num_comv, sizeof(u32)); +} + +static void iris_hfi_gen2_get_buffer(struct iris_buffer *buffer, struct iris_hfi_buffer *buf) +{ + memset(buf, 0, sizeof(*buf)); + buf->type = iris_hfi_gen2_buf_type_from_driver(buffer->type); + buf->index = buffer->index; + buf->base_address = buffer->device_addr; + buf->addr_offset = 0; + buf->buffer_size = buffer->buffer_size; + + if (buffer->type == BUF_INPUT) + buf->buffer_size = ALIGN(buffer->buffer_size, 256); + buf->data_offset = buffer->data_offset; + buf->data_size = buffer->data_size; + if (buffer->attr & BUF_ATTR_PENDING_RELEASE) + buf->flags |= HFI_BUF_HOST_FLAG_RELEASE; + buf->flags |= HFI_BUF_HOST_FLAGS_CB_NON_SECURE; + buf->timestamp = buffer->timestamp; +} + +static int iris_hfi_gen2_session_queue_buffer(struct iris_inst *inst, struct iris_buffer *buffer) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + struct iris_hfi_buffer hfi_buffer; + u32 port; + int ret; + + iris_hfi_gen2_get_buffer(buffer, &hfi_buffer); + if (buffer->type == BUF_COMV) { + ret = iris_set_num_comv(inst); + if (ret) + return ret; + } + + port = iris_hfi_gen2_get_port_from_buf_type(buffer->type); + iris_hfi_gen2_packet_session_command(inst, + HFI_CMD_BUFFER, + HFI_HOST_FLAGS_INTR_REQUIRED, + port, + inst->session_id, + HFI_PAYLOAD_STRUCTURE, + &hfi_buffer, + sizeof(hfi_buffer)); + + return iris_hfi_queue_cmd_write(inst->core, inst_hfi_gen2->packet, + inst_hfi_gen2->packet->size); +} + +static int iris_hfi_gen2_session_release_buffer(struct iris_inst *inst, struct iris_buffer *buffer) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + struct iris_hfi_buffer hfi_buffer; + u32 port; + + iris_hfi_gen2_get_buffer(buffer, &hfi_buffer); + hfi_buffer.flags |= HFI_BUF_HOST_FLAG_RELEASE; + port = iris_hfi_gen2_get_port_from_buf_type(buffer->type); + + iris_hfi_gen2_packet_session_command(inst, + HFI_CMD_BUFFER, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED), + port, + inst->session_id, + HFI_PAYLOAD_STRUCTURE, + &hfi_buffer, + sizeof(hfi_buffer)); + + return iris_hfi_queue_cmd_write(inst->core, inst_hfi_gen2->packet, + inst_hfi_gen2->packet->size); +} + static const struct iris_hfi_command_ops iris_hfi_gen2_command_ops = { .sys_init = iris_hfi_gen2_sys_init, .sys_image_version = iris_hfi_gen2_sys_image_version, @@ -768,6 +904,8 @@ static const struct iris_hfi_command_ops iris_hfi_gen2_command_ops = { .session_set_config_params = iris_hfi_gen2_session_set_config_params, .session_set_property = iris_hfi_gen2_session_set_property, .session_start = iris_hfi_gen2_session_start, + .session_queue_buf = iris_hfi_gen2_session_queue_buffer, + .session_release_buf = iris_hfi_gen2_session_release_buffer, .session_stop = iris_hfi_gen2_session_stop, .session_close = iris_hfi_gen2_session_close, }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h index 92c44841c67d..d9a903ee1e06 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h @@ -17,6 +17,7 @@ #define HFI_CMD_CLOSE 0x01000004 #define HFI_CMD_START 0x01000005 #define HFI_CMD_STOP 0x01000006 +#define HFI_CMD_BUFFER 0x01000009 #define HFI_CMD_SUBSCRIBE_MODE 0x0100000B #define HFI_CMD_END 0x01FFFFFF @@ -53,6 +54,7 @@ #define HFI_PROP_DEC_DEFAULT_HEADER 0x03000168 #define HFI_PROP_DEC_START_FROM_RAP_FRAME 0x03000169 #define HFI_PROP_NO_OUTPUT 0x0300016a +#define HFI_PROP_COMV_BUFFER_COUNT 0x03000193 #define HFI_PROP_END 0x03FFFFFF #define HFI_SESSION_ERROR_BEGIN 0x04000000 @@ -105,6 +107,13 @@ enum hfi_buffer_type { HFI_BUFFER_VPSS = 0x0000000D, }; +enum hfi_buffer_host_flags { + HFI_BUF_HOST_FLAG_RELEASE = 0x00000001, + HFI_BUF_HOST_FLAG_READONLY = 0x00000010, + HFI_BUF_HOST_FLAG_CODEC_CONFIG = 0x00000100, + HFI_BUF_HOST_FLAGS_CB_NON_SECURE = 0x00000200, +}; + enum hfi_packet_firmware_flags { HFI_FW_FLAGS_SUCCESS = 0x00000001, HFI_FW_FLAGS_INFORMATION = 0x00000002, diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h index bb86abd25adb..3083a5174cdc 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_packet.h @@ -61,6 +61,47 @@ struct iris_hfi_packet { u32 payload[]; }; +/** + * struct iris_hfi_buffer + * + * @type: buffer type indicated by "enum hfi_buffer_type" + * FW needs to return proper type for any buffer command. + * @index: index of the buffer + * @base_address: base address of the buffer. + * This buffer address is always 4KBytes aligned. + * @addr_offset: accessible buffer offset from base address + * Decoder bitstream buffer: 256 Bytes aligned + * Firmware can uniquely identify a buffer based on + * base_address & addr_offset. + * HW can read memory only from base_address+addr_offset. + * @buffer_size: accessible buffer size in bytes starting from addr_offset + * @data_offset: data starts from "base_address + addr_offset + data_offset" + * RAW buffer: data_offset is 0. Restriction: 4KBytes aligned + * decoder bitstream buffer: no restriction (can be any value) + * @data_size: data size in bytes + * @flags: buffer flags. It is represented as bit masks. + * host buffer flags are "enum hfi_buffer_host_flags" + * firmware buffer flags are "enum hfi_buffer_firmware_flags" + * @timestamp: timestamp of the buffer in nano seconds (ns) + * It is Presentation timestamp (PTS) for encoder & decoder. + * Decoder: it is pass through from bitstream to raw buffer. + * firmware does not need to return as part of input buffer done. + * For any internal buffers: there is no timestamp. Host sets as 0. + * @reserved: reserved for future use + */ +struct iris_hfi_buffer { + u32 type; + u32 index; + u64 base_address; + u32 addr_offset; + u32 buffer_size; + u32 data_offset; + u32 data_size; + u64 timestamp; + u32 flags; + u32 reserved[5]; +}; + u32 iris_hfi_gen2_get_color_primaries(u32 primaries); u32 iris_hfi_gen2_get_transfer_char(u32 characterstics); u32 iris_hfi_gen2_get_matrix_coefficients(u32 coefficients); diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c index d1a26c18971f..c7f1b6cf102f 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c @@ -25,6 +25,93 @@ struct iris_hfi_gen2_packet_handle { int (*handle)(struct iris_inst *inst, struct iris_hfi_packet *pkt); }; +static u32 iris_hfi_gen2_buf_type_to_driver(enum hfi_buffer_type buf_type) +{ + switch (buf_type) { + case HFI_BUFFER_BITSTREAM: + return BUF_INPUT; + case HFI_BUFFER_RAW: + return BUF_OUTPUT; + case HFI_BUFFER_BIN: + return BUF_BIN; + case HFI_BUFFER_ARP: + return BUF_ARP; + case HFI_BUFFER_COMV: + return BUF_COMV; + case HFI_BUFFER_NON_COMV: + return BUF_NON_COMV; + case HFI_BUFFER_LINE: + return BUF_LINE; + case HFI_BUFFER_DPB: + return BUF_DPB; + case HFI_BUFFER_PERSIST: + return BUF_PERSIST; + default: + return 0; + } +} + +static bool iris_hfi_gen2_is_valid_hfi_buffer_type(u32 buffer_type) +{ + switch (buffer_type) + case HFI_BUFFER_BITSTREAM: + case HFI_BUFFER_RAW: + case HFI_BUFFER_BIN: + case HFI_BUFFER_ARP: + case HFI_BUFFER_COMV: + case HFI_BUFFER_NON_COMV: + case HFI_BUFFER_LINE: + case HFI_BUFFER_DPB: + case HFI_BUFFER_PERSIST: + case HFI_BUFFER_VPSS: + return true; + + return true; +} + +static bool iris_hfi_gen2_is_valid_hfi_port(u32 port, u32 buffer_type) +{ + if (port == HFI_PORT_NONE && buffer_type != HFI_BUFFER_PERSIST) + return false; + + if (port != HFI_PORT_BITSTREAM && port != HFI_PORT_RAW) + return false; + + return true; +} + +static bool iris_hfi_gen2_validate_packet_payload(struct iris_hfi_packet *pkt) +{ + u32 payload_size = 0; + + switch (pkt->payload_info) { + case HFI_PAYLOAD_U32: + case HFI_PAYLOAD_S32: + case HFI_PAYLOAD_Q16: + case HFI_PAYLOAD_U32_ENUM: + case HFI_PAYLOAD_32_PACKED: + payload_size = 4; + break; + case HFI_PAYLOAD_U64: + case HFI_PAYLOAD_S64: + case HFI_PAYLOAD_64_PACKED: + payload_size = 8; + break; + case HFI_PAYLOAD_STRUCTURE: + if (pkt->type == HFI_CMD_BUFFER) + payload_size = sizeof(struct iris_hfi_buffer); + break; + default: + payload_size = 0; + break; + } + + if (pkt->size < sizeof(struct iris_hfi_packet) + payload_size) + return false; + + return true; +} + static int iris_hfi_gen2_validate_packet(u8 *response_pkt, u8 *core_resp_pkt) { u32 response_pkt_size = 0; @@ -141,9 +228,65 @@ static void iris_hfi_gen2_handle_session_close(struct iris_inst *inst, complete(&inst->completion); } +static int iris_hfi_gen2_handle_release_internal_buffer(struct iris_inst *inst, + struct iris_hfi_buffer *buffer) +{ + struct iris_buffer *buf, *iter; + struct iris_buffers *buffers; + u32 buf_type; + int ret = 0; + bool found; + + buf_type = iris_hfi_gen2_buf_type_to_driver(buffer->type); + buffers = &inst->buffers[buf_type]; + + found = false; + list_for_each_entry(iter, &buffers->list, list) { + if (iter->device_addr == buffer->base_address) { + found = true; + buf = iter; + break; + } + } + if (!found) + return -EINVAL; + + buf->attr &= ~BUF_ATTR_QUEUED; + + if (buf->attr & BUF_ATTR_PENDING_RELEASE) + ret = iris_destroy_internal_buffer(inst, buf); + + return ret; +} + +static int iris_hfi_gen2_handle_session_buffer(struct iris_inst *inst, + struct iris_hfi_packet *pkt) +{ + struct iris_hfi_buffer *buffer; + + if (pkt->payload_info == HFI_PAYLOAD_NONE) + return 0; + + if (!iris_hfi_gen2_validate_packet_payload(pkt)) { + iris_inst_change_state(inst, IRIS_INST_ERROR); + return 0; + } + + buffer = (struct iris_hfi_buffer *)((u8 *)pkt + sizeof(*pkt)); + if (!iris_hfi_gen2_is_valid_hfi_buffer_type(buffer->type)) + return 0; + + if (!iris_hfi_gen2_is_valid_hfi_port(pkt->port, buffer->type)) + return 0; + + return iris_hfi_gen2_handle_release_internal_buffer(inst, buffer); +} + static int iris_hfi_gen2_handle_session_command(struct iris_inst *inst, struct iris_hfi_packet *pkt) { + int ret = 0; + switch (pkt->type) { case HFI_CMD_CLOSE: iris_hfi_gen2_handle_session_close(inst, pkt); @@ -151,11 +294,14 @@ static int iris_hfi_gen2_handle_session_command(struct iris_inst *inst, case HFI_CMD_STOP: complete(&inst->completion); break; + case HFI_CMD_BUFFER: + ret = iris_hfi_gen2_handle_session_buffer(inst, pkt); + break; default: break; } - return 0; + return ret; } static int iris_hfi_gen2_handle_session_property(struct iris_inst *inst, diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h index 5a01391845c6..30bc3c0e57a2 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_common.h +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h @@ -68,6 +68,7 @@ enum platform_inst_driver_cap_type { FRAME_WIDTH = 1, FRAME_HEIGHT, MBPF, + NUM_COMV, INST_DRIVER_CAP_MAX, }; @@ -163,6 +164,10 @@ struct iris_platform_data { unsigned int dec_input_prop_size; const u32 *dec_output_prop; unsigned int dec_output_prop_size; + const u32 *dec_ip_int_buf_tbl; + unsigned int dec_ip_int_buf_tbl_size; + const u32 *dec_op_int_buf_tbl; + unsigned int dec_op_int_buf_tbl_size; }; #endif diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c index 6b5444b71090..8316f7081ce0 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c @@ -105,6 +105,15 @@ static const u32 sm8250_vdec_input_config_param[] = { HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE, }; +static const u32 sm8250_dec_ip_int_buf_tbl[] = { + BUF_BIN, + BUF_SCRATCH_1, +}; + +static const u32 sm8250_dec_op_int_buf_tbl[] = { + BUF_DPB, +}; + struct iris_platform_data sm8250_data = { .get_instance = iris_hfi_gen1_get_instance, .init_hfi_command_ops = &iris_hfi_gen1_command_ops_init, @@ -136,4 +145,9 @@ struct iris_platform_data sm8250_data = { sm8250_vdec_input_config_param, .input_config_params_size = ARRAY_SIZE(sm8250_vdec_input_config_param), + + .dec_ip_int_buf_tbl = sm8250_dec_ip_int_buf_tbl, + .dec_ip_int_buf_tbl_size = ARRAY_SIZE(sm8250_dec_ip_int_buf_tbl), + .dec_op_int_buf_tbl = sm8250_dec_op_int_buf_tbl, + .dec_op_int_buf_tbl_size = ARRAY_SIZE(sm8250_dec_op_int_buf_tbl), }; diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c index fccee5309d81..3d84e1a6796c 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c @@ -32,6 +32,12 @@ static struct platform_inst_driver_cap instance_driver_cap_data_sm8550[] = { .max = 138240, .value = 138240, }, + { + .cap_id = NUM_COMV, + .min = 0, + .max = INT_MAX, + .value = 0, + }, }; static struct platform_inst_fw_cap instance_fw_cap_data_sm8550[] = { @@ -216,6 +222,17 @@ static const u32 sm8550_vdec_subscribe_output_properties[] = { HFI_PROP_CABAC_SESSION, }; +static const u32 sm8550_dec_ip_int_buf_tbl[] = { + BUF_BIN, + BUF_COMV, + BUF_NON_COMV, + BUF_LINE, +}; + +static const u32 sm8550_dec_op_int_buf_tbl[] = { + BUF_DPB, +}; + struct iris_platform_data sm8550_data = { .get_instance = iris_hfi_gen2_get_instance, .init_hfi_command_ops = iris_hfi_gen2_command_ops_init, @@ -257,4 +274,9 @@ struct iris_platform_data sm8550_data = { .dec_input_prop_size = ARRAY_SIZE(sm8550_vdec_subscribe_input_properties), .dec_output_prop = sm8550_vdec_subscribe_output_properties, .dec_output_prop_size = ARRAY_SIZE(sm8550_vdec_subscribe_output_properties), + + .dec_ip_int_buf_tbl = sm8550_dec_ip_int_buf_tbl, + .dec_ip_int_buf_tbl_size = ARRAY_SIZE(sm8550_dec_ip_int_buf_tbl), + .dec_op_int_buf_tbl = sm8550_dec_op_int_buf_tbl, + .dec_op_int_buf_tbl_size = ARRAY_SIZE(sm8550_dec_op_int_buf_tbl), }; diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index ae39dd9e0dae..d7895fa4b87c 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -306,7 +306,29 @@ int iris_vdec_streamon_input(struct iris_inst *inst) if (ret) return ret; - return iris_vdec_process_streamon_input(inst); + ret = iris_alloc_and_queue_persist_bufs(inst); + if (ret) + return ret; + + iris_get_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + + ret = iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + + ret = iris_create_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + + ret = iris_queue_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + + ret = iris_vdec_process_streamon_input(inst); + if (ret) + return ret; + + return ret; } static int iris_vdec_process_streamon_output(struct iris_inst *inst) @@ -330,10 +352,24 @@ int iris_vdec_streamon_output(struct iris_inst *inst) if (ret) return ret; + iris_get_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + + ret = iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + if (ret) + goto error; + + ret = iris_create_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + if (ret) + goto error; + ret = iris_vdec_process_streamon_output(inst); if (ret) goto error; + ret = iris_queue_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + if (ret) + return ret; + return ret; error: diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index 2aad96bbaf94..1df724588846 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -168,6 +168,15 @@ int iris_open(struct file *filp) mutex_init(&inst->lock); mutex_init(&inst->ctx_q_lock); + + INIT_LIST_HEAD(&inst->buffers[BUF_BIN].list); + INIT_LIST_HEAD(&inst->buffers[BUF_ARP].list); + INIT_LIST_HEAD(&inst->buffers[BUF_COMV].list); + INIT_LIST_HEAD(&inst->buffers[BUF_NON_COMV].list); + INIT_LIST_HEAD(&inst->buffers[BUF_LINE].list); + INIT_LIST_HEAD(&inst->buffers[BUF_DPB].list); + INIT_LIST_HEAD(&inst->buffers[BUF_PERSIST].list); + INIT_LIST_HEAD(&inst->buffers[BUF_SCRATCH_1].list); init_completion(&inst->completion); init_completion(&inst->flush_completion); @@ -254,6 +263,8 @@ int iris_close(struct file *filp) iris_session_close(inst); iris_inst_change_state(inst, IRIS_INST_DEINIT); iris_v4l2_fh_deinit(inst); + iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + iris_destroy_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); iris_remove_session(inst); mutex_unlock(&inst->lock); mutex_destroy(&inst->ctx_q_lock); diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.c b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c index 7efabaebd082..8cc539066070 100644 --- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.c +++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.c @@ -6,7 +6,194 @@ #include "iris_instance.h" #include "iris_vpu_buffer.h" -u32 iris_vpu_dec_dpb_size(struct iris_inst *inst) +static u32 size_h264d_hw_bin_buffer(u32 frame_width, u32 frame_height, u32 num_vpp_pipes) +{ + u32 size_yuv, size_bin_hdr, size_bin_res; + + size_yuv = ((frame_width * frame_height) <= BIN_BUFFER_THRESHOLD) ? + ((BIN_BUFFER_THRESHOLD * 3) >> 1) : + ((frame_width * frame_height * 3) >> 1); + size_bin_hdr = size_yuv * H264_CABAC_HDR_RATIO_HD_TOT; + size_bin_res = size_yuv * H264_CABAC_RES_RATIO_HD_TOT; + size_bin_hdr = ALIGN(size_bin_hdr / num_vpp_pipes, + DMA_ALIGNMENT) * num_vpp_pipes; + size_bin_res = ALIGN(size_bin_res / num_vpp_pipes, + DMA_ALIGNMENT) * num_vpp_pipes; + + return size_bin_hdr + size_bin_res; +} + +static u32 hfi_buffer_bin_h264d(u32 frame_width, u32 frame_height, u32 num_vpp_pipes) +{ + u32 n_aligned_w, n_aligned_h; + + n_aligned_w = ALIGN(frame_width, 16); + n_aligned_h = ALIGN(frame_height, 16); + + return size_h264d_hw_bin_buffer(n_aligned_w, n_aligned_h, + num_vpp_pipes); +} + +static u32 hfi_buffer_comv_h264d(u32 frame_width, u32 frame_height, u32 _comv_bufcount) +{ + u32 frame_width_in_mbs = DIV_ROUND_UP(frame_width, 16); + u32 frame_height_in_mbs = DIV_ROUND_UP(frame_height, 16); + u32 col_mv_aligned_width = (frame_width_in_mbs << 7); + u32 col_zero_aligned_width = (frame_width_in_mbs << 2); + u32 col_zero_size = 0, size_colloc = 0; + + col_mv_aligned_width = + ALIGN(col_mv_aligned_width, 16); + col_zero_aligned_width = + ALIGN(col_zero_aligned_width, 16); + col_zero_size = col_zero_aligned_width * + ((frame_height_in_mbs + 1) >> 1); + col_zero_size = ALIGN(col_zero_size, 64); + col_zero_size <<= 1; + col_zero_size = ALIGN(col_zero_size, 512); + size_colloc = col_mv_aligned_width * ((frame_height_in_mbs + 1) >> 1); + size_colloc = ALIGN(size_colloc, 64); + size_colloc <<= 1; + size_colloc = ALIGN(size_colloc, 512); + size_colloc += (col_zero_size + SIZE_H264D_BUFTAB_T * 2); + + return (size_colloc * (_comv_bufcount)) + 512; +} + +static u32 size_h264d_bse_cmd_buf(u32 frame_height) +{ + u32 height = ALIGN(frame_height, 32); + + return min_t(u32, (DIV_ROUND_UP(height, 16) * 48), H264D_MAX_SLICE) * + SIZE_H264D_BSE_CMD_PER_BUF; +} + +static u32 size_h264d_vpp_cmd_buf(u32 frame_height) +{ + u32 size, height; + + height = ALIGN(frame_height, 32); + size = min_t(u32, (DIV_ROUND_UP(height, 16) * 48), H264D_MAX_SLICE) * + SIZE_H264D_VPP_CMD_PER_BUF; + + if (size > VPP_CMD_MAX_SIZE) + size = VPP_CMD_MAX_SIZE; + + return size; +} + +static u32 hfi_buffer_persist_h264d(void) +{ + return ALIGN(SIZE_SLIST_BUF_H264 * NUM_SLIST_BUF_H264 + + H264_DISPLAY_BUF_SIZE * H264_NUM_FRM_INFO + + NUM_HW_PIC_BUF * SIZE_SEI_USERDATA, + DMA_ALIGNMENT); +} + +static u32 hfi_buffer_non_comv_h264d(u32 frame_width, u32 frame_height, u32 num_vpp_pipes) +{ + u32 size_bse, size_vpp, size; + + size_bse = size_h264d_bse_cmd_buf(frame_height); + size_vpp = size_h264d_vpp_cmd_buf(frame_height); + size = ALIGN(size_bse, DMA_ALIGNMENT) + + ALIGN(size_vpp, DMA_ALIGNMENT) + + ALIGN(SIZE_HW_PIC(SIZE_H264D_HW_PIC_T), DMA_ALIGNMENT); + + return ALIGN(size, DMA_ALIGNMENT); +} + +static u32 size_vpss_lb(u32 frame_width, u32 frame_height, u32 num_vpp_pipes) +{ + u32 vpss_4tap_left_buffer_size = 0, vpss_div2_left_buffer_size = 0; + u32 vpss_4tap_top_buffer_size = 0, vpss_div2_top_buffer_size = 0; + u32 opb_lb_wr_llb_y_buffer_size, opb_lb_wr_llb_uv_buffer_size; + u32 opb_wr_top_line_chroma_buffer_size; + u32 opb_wr_top_line_luma_buffer_size; + u32 macrotiling_size = 32, size; + + opb_wr_top_line_luma_buffer_size = + ALIGN(frame_width, macrotiling_size) / macrotiling_size * 256; + opb_wr_top_line_luma_buffer_size = + ALIGN(opb_wr_top_line_luma_buffer_size, DMA_ALIGNMENT) + + (MAX_TILE_COLUMNS - 1) * 256; + opb_wr_top_line_luma_buffer_size = + max_t(u32, opb_wr_top_line_luma_buffer_size, (32 * ALIGN(frame_height, 8))); + opb_wr_top_line_chroma_buffer_size = opb_wr_top_line_luma_buffer_size; + opb_lb_wr_llb_uv_buffer_size = + ALIGN((ALIGN(frame_height, 8) / (4 / 2)) * 64, 32); + opb_lb_wr_llb_y_buffer_size = + ALIGN((ALIGN(frame_height, 8) / (4 / 2)) * 64, 32); + size = num_vpp_pipes * 2 * + (vpss_4tap_top_buffer_size + vpss_div2_top_buffer_size) + + 2 * (vpss_4tap_left_buffer_size + vpss_div2_left_buffer_size) + + opb_wr_top_line_luma_buffer_size + + opb_wr_top_line_chroma_buffer_size + + opb_lb_wr_llb_uv_buffer_size + + opb_lb_wr_llb_y_buffer_size; + + return size; +} + +static u32 hfi_buffer_line_h264d(u32 frame_width, u32 frame_height, + bool is_opb, u32 num_vpp_pipes) +{ + u32 vpss_lb_size = 0; + u32 size; + + size = ALIGN(size_h264d_lb_fe_top_data(frame_width), DMA_ALIGNMENT) + + ALIGN(size_h264d_lb_fe_top_ctrl(frame_width), DMA_ALIGNMENT) + + ALIGN(size_h264d_lb_fe_left_ctrl(frame_height), DMA_ALIGNMENT) * num_vpp_pipes + + ALIGN(size_h264d_lb_se_top_ctrl(frame_width), DMA_ALIGNMENT) + + ALIGN(size_h264d_lb_se_left_ctrl(frame_height), DMA_ALIGNMENT) * num_vpp_pipes + + ALIGN(size_h264d_lb_pe_top_data(frame_width), DMA_ALIGNMENT) + + ALIGN(size_h264d_lb_vsp_top(frame_width), DMA_ALIGNMENT) + + ALIGN(size_h264d_lb_recon_dma_metadata_wr(frame_height), DMA_ALIGNMENT) * 2 + + ALIGN(size_h264d_qp(frame_width, frame_height), DMA_ALIGNMENT); + size = ALIGN(size, DMA_ALIGNMENT); + if (is_opb) + vpss_lb_size = size_vpss_lb(frame_width, frame_height, num_vpp_pipes); + + size = ALIGN((size + vpss_lb_size), DMA_ALIGNMENT); + + return size; +} + +static u32 iris_vpu_dec_bin_size(struct iris_inst *inst) +{ + struct iris_core *core = inst->core; + u32 width, height, num_vpp_pipes; + struct v4l2_format *f; + + num_vpp_pipes = core->iris_platform_data->num_vpp_pipe; + + f = inst->fmt_src; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + return hfi_buffer_bin_h264d(width, height, num_vpp_pipes); +} + +static u32 iris_vpu_dec_comv_size(struct iris_inst *inst) +{ + u32 width, height, num_comv; + struct v4l2_format *f; + + f = inst->fmt_src; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + num_comv = inst->buffers[BUF_OUTPUT].min_count; + + return hfi_buffer_comv_h264d(width, height, num_comv); +} + +static u32 iris_vpu_dec_persist_size(struct iris_inst *inst) +{ + return hfi_buffer_persist_h264d(); +} + +static u32 iris_vpu_dec_dpb_size(struct iris_inst *inst) { if (iris_split_mode_enabled(inst)) return iris_get_buffer_size(inst, BUF_DPB); @@ -14,6 +201,82 @@ u32 iris_vpu_dec_dpb_size(struct iris_inst *inst) return 0; } +static u32 iris_vpu_dec_non_comv_size(struct iris_inst *inst) +{ + struct iris_core *core = inst->core; + u32 width, height, num_vpp_pipes; + struct v4l2_format *f; + + num_vpp_pipes = core->iris_platform_data->num_vpp_pipe; + + f = inst->fmt_src; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + return hfi_buffer_non_comv_h264d(width, height, num_vpp_pipes); +} + +static u32 iris_vpu_dec_line_size(struct iris_inst *inst) +{ + struct iris_core *core = inst->core; + u32 width, height, num_vpp_pipes; + struct v4l2_format *f; + bool is_opb = false; + + num_vpp_pipes = core->iris_platform_data->num_vpp_pipe; + + if (iris_split_mode_enabled(inst)) + is_opb = true; + + f = inst->fmt_src; + width = f->fmt.pix_mp.width; + height = f->fmt.pix_mp.height; + + return hfi_buffer_line_h264d(width, height, is_opb, + num_vpp_pipes); +} + +static u32 iris_vpu_dec_scratch1_size(struct iris_inst *inst) +{ + return iris_vpu_dec_comv_size(inst) + + iris_vpu_dec_non_comv_size(inst) + + iris_vpu_dec_line_size(inst); +} + +struct iris_vpu_buf_type_handle { + enum iris_buffer_type type; + u32 (*handle)(struct iris_inst *inst); +}; + +int iris_vpu_buf_size(struct iris_inst *inst, enum iris_buffer_type buffer_type) +{ + const struct iris_vpu_buf_type_handle *buf_type_handle_arr = NULL; + u32 size = 0, buf_type_handle_size = 0; + int i; + + static const struct iris_vpu_buf_type_handle dec_internal_buf_type_handle[] = { + {BUF_BIN, iris_vpu_dec_bin_size }, + {BUF_COMV, iris_vpu_dec_comv_size }, + {BUF_NON_COMV, iris_vpu_dec_non_comv_size }, + {BUF_LINE, iris_vpu_dec_line_size }, + {BUF_PERSIST, iris_vpu_dec_persist_size }, + {BUF_DPB, iris_vpu_dec_dpb_size }, + {BUF_SCRATCH_1, iris_vpu_dec_scratch1_size }, + }; + + buf_type_handle_size = ARRAY_SIZE(dec_internal_buf_type_handle); + buf_type_handle_arr = dec_internal_buf_type_handle; + + for (i = 0; i < buf_type_handle_size; i++) { + if (buf_type_handle_arr[i].type == buffer_type) { + size = buf_type_handle_arr[i].handle(inst); + break; + } + } + + return size; +} + static inline int iris_vpu_dpb_count(struct iris_inst *inst) { int count = 0; @@ -33,6 +296,13 @@ int iris_vpu_buf_count(struct iris_inst *inst, enum iris_buffer_type buffer_type return MIN_BUFFERS; case BUF_OUTPUT: return inst->fw_min_count; + case BUF_BIN: + case BUF_COMV: + case BUF_NON_COMV: + case BUF_LINE: + case BUF_PERSIST: + case BUF_SCRATCH_1: + return 1; /* internal buffer count needed by firmware is 1 */ case BUF_DPB: return iris_vpu_dpb_count(inst); default: diff --git a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h index 03a181c5754d..1bc598ee9c20 100644 --- a/drivers/media/platform/qcom/iris/iris_vpu_buffer.h +++ b/drivers/media/platform/qcom/iris/iris_vpu_buffer.h @@ -10,7 +10,82 @@ struct iris_inst; #define MIN_BUFFERS 4 -u32 iris_vpu_dec_dpb_size(struct iris_inst *inst); +#define DMA_ALIGNMENT 256 + +#define NUM_HW_PIC_BUF 32 +#define SIZE_HW_PIC(size_per_buf) (NUM_HW_PIC_BUF * (size_per_buf)) + +#define MAX_TILE_COLUMNS 32 +#define BIN_BUFFER_THRESHOLD (1280 * 736) +#define VPP_CMD_MAX_SIZE (BIT(20)) +#define H264D_MAX_SLICE 1800 + +#define SIZE_H264D_BUFTAB_T 256 +#define SIZE_H264D_BSE_CMD_PER_BUF (32 * 4) +#define SIZE_H264D_VPP_CMD_PER_BUF 512 + +#define NUM_SLIST_BUF_H264 (256 + 32) +#define SIZE_SLIST_BUF_H264 512 +#define H264_DISPLAY_BUF_SIZE 3328 +#define H264_NUM_FRM_INFO 66 + +#define SIZE_SEI_USERDATA 4096 + +#define H264_CABAC_HDR_RATIO_HD_TOT 1 +#define H264_CABAC_RES_RATIO_HD_TOT 3 +#define SIZE_H264D_HW_PIC_T (BIT(11)) + +#define MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE 64 +#define MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE 16 +#define MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE 384 +#define MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE 640 + +static inline u32 size_h264d_lb_fe_top_data(u32 frame_width) +{ + return MAX_FE_NBR_DATA_LUMA_LINE_BUFFER_SIZE * ALIGN(frame_width, 16) * 3; +} + +static inline u32 size_h264d_lb_fe_top_ctrl(u32 frame_width) +{ + return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * DIV_ROUND_UP(frame_width, 16); +} + +static inline u32 size_h264d_lb_fe_left_ctrl(u32 frame_height) +{ + return MAX_FE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * DIV_ROUND_UP(frame_height, 16); +} + +static inline u32 size_h264d_lb_se_top_ctrl(u32 frame_width) +{ + return MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * DIV_ROUND_UP(frame_width, 16); +} + +static inline u32 size_h264d_lb_se_left_ctrl(u32 frame_height) +{ + return MAX_SE_NBR_CTRL_LCU64_LINE_BUFFER_SIZE * DIV_ROUND_UP(frame_height, 16); +} + +static inline u32 size_h264d_lb_pe_top_data(u32 frame_width) +{ + return MAX_PE_NBR_DATA_LCU64_LINE_BUFFER_SIZE * DIV_ROUND_UP(frame_width, 16); +} + +static inline u32 size_h264d_lb_vsp_top(u32 frame_width) +{ + return (DIV_ROUND_UP(frame_width, 16) << 7); +} + +static inline u32 size_h264d_lb_recon_dma_metadata_wr(u32 frame_height) +{ + return ALIGN(frame_height, 16) * 32; +} + +static inline u32 size_h264d_qp(u32 frame_width, u32 frame_height) +{ + return DIV_ROUND_UP(frame_width, 64) * DIV_ROUND_UP(frame_height, 64) * 128; +} + +int iris_vpu_buf_size(struct iris_inst *inst, enum iris_buffer_type buffer_type); int iris_vpu_buf_count(struct iris_inst *inst, enum iris_buffer_type buffer_type); #endif From patchwork Tue Aug 27 10:05:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 822719 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7752E1A08B1; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; cv=none; b=jEKAeUNvUqJLL6AV7QlwEUgCG7c+LuG23xVPwpqLPS+QrJHlh3DdCmtDLL3yYv89lmWAmdu6fpsNTQ71ufOPd6vPpK73+eSIz1etRJZk40I8F7FJ9+JSLZhIGTqj9RV65O64+mVUDcRwDgBJee9fMBuSXPhVw959Devmqm8GKxg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; c=relaxed/simple; bh=Jktsc4+XkkJiJ7GPYzKUtgSH15yBHyp8CsMXKASnWjw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=D4bdUxtJMESSjrPREWBU2JIemmsx75nirQNf/u70CKB5k8XWmLJvBUZHVupiD0eWMZxLIyPB6eZO2fN3kexad3yLFlsGAj9Ir0pSNim6mTLVRjYhs4lJCj96QufPsSPKpFX7HQUCZgXHaF/A/kKh/fQbZrdeOnM6NJuJ9FvINOQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lu4SglL4; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lu4SglL4" Received: by smtp.kernel.org (Postfix) with ESMTPS id 4D6F7C4AF61; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753196; bh=Jktsc4+XkkJiJ7GPYzKUtgSH15yBHyp8CsMXKASnWjw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=lu4SglL4qIZyyb21vnR5u8KkpYm9SR3FwaQ2bCfqe7T+ZiomN1IbZe0crhYvvo3fA ysNvvtwJGNlVUrlvtQYZkJ70U/kB/cyKgXx83wa6p1MnBYduqnOJnJXEuiUHc+wlbx 7dgiEACe+f7ztsfu+F7d9nE+98ZqbRIj/pXTJVKzaQVRqYlIw1pGuTKY6rsnjIISkV rAg//pTpz10iv7ATsuFJIzc4jbSZSfpGWbBM0w3pk5DJ++NoxsYCL8MWbgViRrSWRR M7kuM7dn4ueR39qDUe9mXubGrs0O1IciJE7YqerTMFUpWdH/wzrHqGi4jBMcXbK7hR +R56nONL0V0Lg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FCC5C54735; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:47 +0530 Subject: [PATCH v3 22/29] media: iris: implement vb2 ops for buf_queue and firmware response Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-22-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=43514; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=PtNAnaCUiK3M62TveAlkDRKHkx76I9vCsxoGucFGC/8=; b=tPt5LEg73KTMrXCMuz6QkPYiLBQHgJGSKrQbiAlX2uDwaUMSs4nrzCtpz0/MayOvzllAAWU2t 7RvqgMtLyaDAVDmHwlGNXd6nPYVDNiXztg5Yv8rkmGIDOD+VuTSUpar X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal Implement vb2 ops for buf queue. Below are the different buffer attributes: BUF_ATTR_DEFERRED - buffer queued by client but not submitted to firmware. BUF_ATTR_PENDING_RELEASE - buffers requested to be released from firmware. BUF_ATTR_QUEUED - buffers submitted to firmware. BUF_ATTR_DEQUEUED - buffers received from firmware. BUF_ATTR_BUFFER_DONE - buffers sent back to vb2. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/Kconfig | 1 + drivers/media/platform/qcom/iris/iris_buffer.c | 82 +++++++++ drivers/media/platform/qcom/iris/iris_buffer.h | 2 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 53 ++++++ .../platform/qcom/iris/iris_hfi_gen1_defines.h | 80 +++++++++ .../platform/qcom/iris/iris_hfi_gen1_response.c | 170 ++++++++++++++++++- drivers/media/platform/qcom/iris/iris_hfi_gen2.h | 2 + .../platform/qcom/iris/iris_hfi_gen2_defines.h | 22 +++ .../platform/qcom/iris/iris_hfi_gen2_response.c | 185 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_utils.c | 16 ++ drivers/media/platform/qcom/iris/iris_utils.h | 9 + drivers/media/platform/qcom/iris/iris_vb2.c | 70 +++++++- drivers/media/platform/qcom/iris/iris_vb2.h | 2 + drivers/media/platform/qcom/iris/iris_vdec.c | 129 +++++++++++++- drivers/media/platform/qcom/iris/iris_vdec.h | 3 +- drivers/media/platform/qcom/iris/iris_vidc.c | 11 ++ 16 files changed, 825 insertions(+), 12 deletions(-) diff --git a/drivers/media/platform/qcom/iris/Kconfig b/drivers/media/platform/qcom/iris/Kconfig index ea518760c65c..dbec672ccf37 100644 --- a/drivers/media/platform/qcom/iris/Kconfig +++ b/drivers/media/platform/qcom/iris/Kconfig @@ -5,6 +5,7 @@ config VIDEO_QCOM_IRIS select V4L2_MEM2MEM_DEV select QCOM_MDT_LOADER if ARCH_QCOM select QCOM_SCM + select VIDEOBUF2_DMA_CONTIG help This is a V4L2 driver for Qualcomm Iris video accelerator hardware. It accelerates decoding operations on various diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c b/drivers/media/platform/qcom/iris/iris_buffer.c index 57f3ffd8596a..ed97e9f145a5 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/iris/iris_buffer.c @@ -3,6 +3,7 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include #include #include "iris_buffer.h" @@ -439,6 +440,36 @@ int iris_alloc_and_queue_persist_bufs(struct iris_inst *inst) return ret; } +int iris_queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_type) +{ + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + struct v4l2_m2m_buffer *buffer, *n; + struct iris_buffer *buf = NULL; + int ret; + + if (buf_type == BUF_INPUT) { + v4l2_m2m_for_each_src_buf_safe(m2m_ctx, buffer, n) { + buf = to_iris_buffer(&buffer->vb); + if (!(buf->attr & BUF_ATTR_DEFERRED)) + continue; + ret = iris_queue_buffer(inst, buf); + if (ret) + return ret; + } + } else { + v4l2_m2m_for_each_dst_buf_safe(m2m_ctx, buffer, n) { + buf = to_iris_buffer(&buffer->vb); + if (!(buf->attr & BUF_ATTR_DEFERRED)) + continue; + ret = iris_queue_buffer(inst, buf); + if (ret) + return ret; + } + } + + return 0; +} + void iris_vb2_queue_error(struct iris_inst *inst) { struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; @@ -449,3 +480,54 @@ void iris_vb2_queue_error(struct iris_inst *inst) q = v4l2_m2m_get_dst_vq(m2m_ctx); vb2_queue_error(q); } + +static struct vb2_v4l2_buffer * +iris_helper_find_buf(struct iris_inst *inst, unsigned int type, u32 idx) +{ + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + + if (V4L2_TYPE_IS_OUTPUT(type)) + return v4l2_m2m_src_buf_remove_by_idx(m2m_ctx, idx); + else + return v4l2_m2m_dst_buf_remove_by_idx(m2m_ctx, idx); +} + +int iris_vb2_buffer_done(struct iris_inst *inst, struct iris_buffer *buf) +{ + struct vb2_v4l2_buffer *vbuf; + struct vb2_buffer *vb2; + int type, state; + + if (buf->type == BUF_INPUT) + type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; + else if (buf->type == BUF_OUTPUT) + type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + else + return 0; /* Internal DPB Buffers */ + + vbuf = iris_helper_find_buf(inst, type, buf->index); + if (!vbuf) + return -EINVAL; + + vb2 = &vbuf->vb2_buf; + + if (buf->flags & V4L2_BUF_FLAG_ERROR) + state = VB2_BUF_STATE_ERROR; + else + state = VB2_BUF_STATE_DONE; + + vbuf->flags = buf->flags; + + if (vbuf->flags & V4L2_BUF_FLAG_LAST) { + if (inst->subscriptions & V4L2_EVENT_EOS) { + const struct v4l2_event ev = { .type = V4L2_EVENT_EOS }; + + v4l2_event_queue_fh(&inst->fh, &ev); + } + } + vb2->timestamp = buf->timestamp; + vb2->planes[0].bytesused = buf->data_size + vb2->planes[0].data_offset; + v4l2_m2m_buf_done(vbuf, state); + + return 0; +} diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h index 41e090e9f0b0..bd8b74a9fbc4 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.h +++ b/drivers/media/platform/qcom/iris/iris_buffer.h @@ -109,6 +109,8 @@ int iris_destroy_internal_buffer(struct iris_inst *inst, struct iris_buffer *buf int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane); int iris_alloc_and_queue_persist_bufs(struct iris_inst *inst); int iris_queue_buffer(struct iris_inst *inst, struct iris_buffer *buf); +int iris_queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_type); +int iris_vb2_buffer_done(struct iris_inst *inst, struct iris_buffer *buf); void iris_vb2_queue_error(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c index be45a38166eb..5edee12d6a1b 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c @@ -180,6 +180,10 @@ static int iris_hfi_gen1_session_stop(struct iris_inst *inst, u32 plane) ret = iris_hfi_queue_cmd_write(core, &pkt, pkt.shdr.hdr.size); if (!ret) ret = iris_wait_for_session_response(inst, false); + iris_helper_buffers_done(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE, + VB2_BUF_STATE_ERROR); + iris_helper_buffers_done(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE, + VB2_BUF_STATE_ERROR); } else if (inst->state == IRIS_INST_STREAMING) { if (V4L2_TYPE_IS_OUTPUT(plane)) flush_type = HFI_FLUSH_ALL; @@ -201,6 +205,50 @@ static int iris_hfi_gen1_session_stop(struct iris_inst *inst, u32 plane) return ret; } +static int iris_hfi_gen1_queue_input_buffer(struct iris_inst *inst, struct iris_buffer *buf) +{ + struct hfi_session_empty_buffer_compressed_pkt ip_pkt; + + ip_pkt.shdr.hdr.size = sizeof(struct hfi_session_empty_buffer_compressed_pkt); + ip_pkt.shdr.hdr.pkt_type = HFI_CMD_SESSION_EMPTY_BUFFER; + ip_pkt.shdr.session_id = inst->session_id; + ip_pkt.time_stamp_hi = upper_32_bits(buf->timestamp); + ip_pkt.time_stamp_lo = lower_32_bits(buf->timestamp); + ip_pkt.flags = buf->flags; + ip_pkt.mark_target = 0; + ip_pkt.mark_data = 0; + ip_pkt.offset = buf->data_offset; + ip_pkt.alloc_len = buf->buffer_size; + ip_pkt.filled_len = buf->data_size; + ip_pkt.input_tag = buf->index; + ip_pkt.packet_buffer = buf->device_addr; + + return iris_hfi_queue_cmd_write(inst->core, &ip_pkt, ip_pkt.shdr.hdr.size); +} + +static int iris_hfi_gen1_queue_output_buffer(struct iris_inst *inst, struct iris_buffer *buf) +{ + struct hfi_session_fill_buffer_pkt op_pkt; + + op_pkt.shdr.hdr.size = sizeof(struct hfi_session_fill_buffer_pkt); + op_pkt.shdr.hdr.pkt_type = HFI_CMD_SESSION_FILL_BUFFER; + op_pkt.shdr.session_id = inst->session_id; + op_pkt.output_tag = buf->index; + op_pkt.packet_buffer = buf->device_addr; + op_pkt.extradata_buffer = 0; + op_pkt.alloc_len = buf->buffer_size; + op_pkt.filled_len = buf->data_size; + op_pkt.offset = buf->data_offset; + op_pkt.data = 0; + + if (buf->type == BUF_OUTPUT && iris_split_mode_enabled(inst)) + op_pkt.stream_id = 1; + else + op_pkt.stream_id = 0; + + return iris_hfi_queue_cmd_write(inst->core, &op_pkt, op_pkt.shdr.hdr.size); +} + static int iris_hfi_gen1_queue_internal_buffer(struct iris_inst *inst, struct iris_buffer *buf) { struct hfi_session_set_buffers_pkt *int_pkt; @@ -240,6 +288,11 @@ static int iris_hfi_gen1_queue_internal_buffer(struct iris_inst *inst, struct ir static int iris_hfi_gen1_session_queue_buffer(struct iris_inst *inst, struct iris_buffer *buf) { switch (buf->type) { + case BUF_INPUT: + return iris_hfi_gen1_queue_input_buffer(inst, buf); + case BUF_OUTPUT: + case BUF_DPB: + return iris_hfi_gen1_queue_output_buffer(inst, buf); case BUF_PERSIST: case BUF_BIN: case BUF_SCRATCH_1: diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h index 5527233b03e5..b7f9e0f47675 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h @@ -29,11 +29,14 @@ #define HFI_CMD_SESSION_LOAD_RESOURCES 0x211001 #define HFI_CMD_SESSION_START 0x211002 #define HFI_CMD_SESSION_STOP 0x211003 +#define HFI_CMD_SESSION_EMPTY_BUFFER 0x211004 +#define HFI_CMD_SESSION_FILL_BUFFER 0x211005 #define HFI_CMD_SESSION_FLUSH 0x211008 #define HFI_CMD_SESSION_RELEASE_BUFFERS 0x21100b #define HFI_CMD_SESSION_RELEASE_RESOURCES 0x21100c #define HFI_ERR_SESSION_UNSUPPORTED_SETTING 0x1008 +#define HFI_ERR_SESSION_UNSUPPORTED_STREAM 0x100d #define HFI_ERR_SESSION_UNSUPPORT_BUFFERTYPE 0x1010 #define HFI_ERR_SESSION_INVALID_SCALE_FACTOR 0x1012 #define HFI_ERR_SESSION_UPSCALE_NOT_SUPPORTED 0x1013 @@ -41,6 +44,8 @@ #define HFI_EVENT_SYS_ERROR 0x1 #define HFI_EVENT_SESSION_ERROR 0x2 +#define HFI_BUFFERFLAG_TIMESTAMPINVALID 0x00000100 + #define HFI_FLUSH_OUTPUT 0x1000002 #define HFI_FLUSH_OUTPUT2 0x1000003 #define HFI_FLUSH_ALL 0x1000004 @@ -84,9 +89,19 @@ #define HFI_MSG_SESSION_START 0x221002 #define HFI_MSG_SESSION_STOP 0x221003 #define HFI_MSG_SESSION_FLUSH 0x221006 +#define HFI_MSG_SESSION_EMPTY_BUFFER 0x221007 +#define HFI_MSG_SESSION_FILL_BUFFER 0x221008 #define HFI_MSG_SESSION_RELEASE_RESOURCES 0x22100a #define HFI_MSG_SESSION_RELEASE_BUFFERS 0x22100c +#define HFI_PICTURE_I 0x00000001 +#define HFI_PICTURE_P 0x00000002 +#define HFI_PICTURE_B 0x00000004 +#define HFI_PICTURE_IDR 0x00000008 +#define HFI_FRAME_NOTCODED 0x7f002000 +#define HFI_FRAME_YUV 0x7f004000 +#define HFI_UNUSED_PICT 0x10000000 + struct hfi_pkt_hdr { u32 size; u32 pkt_type; @@ -144,6 +159,34 @@ struct hfi_session_set_buffers_pkt { u32 buffer_info[]; }; +struct hfi_session_empty_buffer_compressed_pkt { + struct hfi_session_hdr_pkt shdr; + u32 time_stamp_hi; + u32 time_stamp_lo; + u32 flags; + u32 mark_target; + u32 mark_data; + u32 offset; + u32 alloc_len; + u32 filled_len; + u32 input_tag; + u32 packet_buffer; + u32 extradata_buffer; + u32 data; +}; + +struct hfi_session_fill_buffer_pkt { + struct hfi_session_hdr_pkt shdr; + u32 stream_id; + u32 offset; + u32 alloc_len; + u32 filled_len; + u32 output_tag; + u32 packet_buffer; + u32 extradata_buffer; + u32 data; +}; + struct hfi_session_flush_pkt { struct hfi_session_hdr_pkt shdr; u32 flush_type; @@ -258,6 +301,43 @@ struct hfi_multi_stream { u32 enable; }; +struct hfi_msg_session_empty_buffer_done_pkt { + struct hfi_msg_session_hdr_pkt shdr; + u32 offset; + u32 filled_len; + u32 input_tag; + u32 packet_buffer; + u32 extradata_buffer; + u32 data[]; +}; + +struct hfi_msg_session_fbd_uncompressed_plane0_pkt { + struct hfi_session_hdr_pkt shdr; + u32 stream_id; + u32 view_id; + u32 error_type; + u32 time_stamp_hi; + u32 time_stamp_lo; + u32 flags; + u32 mark_target; + u32 mark_data; + u32 stats; + u32 alloc_len; + u32 filled_len; + u32 offset; + u32 frame_width; + u32 frame_height; + u32 start_x_coord; + u32 start_y_coord; + u32 input_tag; + u32 input_tag2; + u32 output_tag; + u32 picture_type; + u32 packet_buffer; + u32 extradata_buffer; + u32 data[]; +}; + struct hfi_msg_session_release_buffers_done_pkt { struct hfi_msg_session_hdr_pkt shdr; u32 num_buffers; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c index af1f59c43b92..32fb01616122 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c @@ -3,6 +3,8 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include + #include "iris_hfi_gen1.h" #include "iris_hfi_gen1_defines.h" #include "iris_instance.h" @@ -123,6 +125,146 @@ static void iris_hfi_gen1_sys_property_info(struct iris_core *core, void *packet } } +static void iris_hfi_gen1_session_etb_done(struct iris_inst *inst, void *packet) +{ + struct hfi_msg_session_empty_buffer_done_pkt *pkt = packet; + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + struct v4l2_m2m_buffer *m2m_buffer, *n; + struct iris_buffer *buf = NULL; + bool found = false; + + v4l2_m2m_for_each_src_buf_safe(m2m_ctx, m2m_buffer, n) { + buf = to_iris_buffer(&m2m_buffer->vb); + if (buf->index == pkt->input_tag) { + found = true; + break; + } + } + if (!found) + goto error; + + if (pkt->shdr.error_type == HFI_ERR_SESSION_UNSUPPORTED_STREAM) { + buf->flags = V4L2_BUF_FLAG_ERROR; + iris_vb2_queue_error(inst); + iris_inst_change_state(inst, IRIS_INST_ERROR); + } + + if (!(buf->attr & BUF_ATTR_QUEUED)) + return; + + buf->data_size = pkt->filled_len; + buf->attr &= ~BUF_ATTR_QUEUED; + + if (!(buf->attr & BUF_ATTR_BUFFER_DONE)) { + buf->attr |= BUF_ATTR_BUFFER_DONE; + iris_vb2_buffer_done(inst, buf); + } + + return; + +error: + iris_inst_change_state(inst, IRIS_INST_ERROR); + dev_err(inst->core->dev, "error in etb done\n"); +} + +static void iris_hfi_gen1_session_ftb_done(struct iris_inst *inst, void *packet) +{ + struct hfi_msg_session_fbd_uncompressed_plane0_pkt *pkt = packet; + u32 flags = 0, hfi_flags = 0, offset = 0, filled_len = 0; + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + struct v4l2_m2m_buffer *m2m_buffer, *n; + u32 timestamp_hi = 0, timestamp_lo = 0; + struct iris_buffer *buf = NULL, *iter; + struct iris_core *core = inst->core; + u32 pic_type = 0, output_tag = -1; + struct iris_buffers *buffers; + u64 timestamp_us = 0; + bool found = false; + + timestamp_hi = pkt->time_stamp_hi; + timestamp_lo = pkt->time_stamp_lo; + hfi_flags = pkt->flags; + offset = pkt->offset; + filled_len = pkt->filled_len; + pic_type = pkt->picture_type; + output_tag = pkt->output_tag; + + if (iris_split_mode_enabled(inst) && pkt->stream_id == 0) { + buffers = &inst->buffers[BUF_DPB]; + if (!buffers) + goto error; + + found = false; + list_for_each_entry(iter, &buffers->list, list) { + if (!(iter->attr & BUF_ATTR_QUEUED)) + continue; + + found = (iter->index == output_tag && + iter->data_offset == offset); + + if (found) { + buf = iter; + break; + } + } + } else { + v4l2_m2m_for_each_dst_buf_safe(m2m_ctx, m2m_buffer, n) { + buf = to_iris_buffer(&m2m_buffer->vb); + if (!(buf->attr & BUF_ATTR_QUEUED)) + continue; + + found = (buf->index == output_tag && + buf->data_offset == offset); + + if (found) + break; + } + } + if (!found) + goto error; + + buf->data_offset = offset; + buf->data_size = filled_len; + + if (!(hfi_flags & HFI_BUFFERFLAG_TIMESTAMPINVALID) && filled_len) { + timestamp_us = timestamp_hi; + timestamp_us = (timestamp_us << 32) | timestamp_lo; + } + buf->timestamp = timestamp_us; + + switch (pic_type) { + case HFI_PICTURE_IDR: + case HFI_PICTURE_I: + flags |= V4L2_BUF_FLAG_KEYFRAME; + break; + case HFI_PICTURE_P: + flags |= V4L2_BUF_FLAG_PFRAME; + break; + case HFI_PICTURE_B: + flags |= V4L2_BUF_FLAG_BFRAME; + break; + case HFI_FRAME_NOTCODED: + case HFI_UNUSED_PICT: + case HFI_FRAME_YUV: + default: + break; + } + + buf->attr &= ~BUF_ATTR_QUEUED; + buf->attr |= BUF_ATTR_DEQUEUED; + buf->attr |= BUF_ATTR_BUFFER_DONE; + + buf->flags = flags; + + iris_vb2_buffer_done(inst, buf); + + return; + +error: + iris_inst_change_state(inst, IRIS_INST_ERROR); + dev_err(core->dev, "error in ftb done\n"); +} + struct iris_hfi_gen1_response_pkt_info { u32 pkt; u32 pkt_sz; @@ -161,6 +303,14 @@ static const struct iris_hfi_gen1_response_pkt_info pkt_infos[] = { .pkt = HFI_MSG_SESSION_STOP, .pkt_sz = sizeof(struct hfi_msg_session_hdr_pkt), }, + { + .pkt = HFI_MSG_SESSION_EMPTY_BUFFER, + .pkt_sz = sizeof(struct hfi_msg_session_empty_buffer_done_pkt), + }, + { + .pkt = HFI_MSG_SESSION_FILL_BUFFER, + .pkt_sz = sizeof(struct hfi_msg_session_fbd_uncompressed_plane0_pkt), + }, { .pkt = HFI_MSG_SESSION_FLUSH, .pkt_sz = sizeof(struct hfi_msg_session_flush_done_pkt), @@ -231,15 +381,21 @@ static void iris_hfi_gen1_handle_response(struct iris_core *core, void *response } mutex_lock(&inst->lock); - struct hfi_msg_session_hdr_pkt *shdr; + if (hdr->pkt_type == HFI_MSG_SESSION_EMPTY_BUFFER) { + iris_hfi_gen1_session_etb_done(inst, hdr); + } else if (hdr->pkt_type == HFI_MSG_SESSION_FILL_BUFFER) { + iris_hfi_gen1_session_ftb_done(inst, hdr); + } else { + struct hfi_msg_session_hdr_pkt *shdr; - shdr = (struct hfi_msg_session_hdr_pkt *)hdr; - if (shdr->error_type != HFI_ERR_NONE) - iris_inst_change_state(inst, IRIS_INST_ERROR); + shdr = (struct hfi_msg_session_hdr_pkt *)hdr; + if (shdr->error_type != HFI_ERR_NONE) + iris_inst_change_state(inst, IRIS_INST_ERROR); - done = pkt_info->pkt == HFI_MSG_SESSION_FLUSH ? - &inst->flush_completion : &inst->completion; - complete(done); + done = pkt_info->pkt == HFI_MSG_SESSION_FLUSH ? + &inst->flush_completion : &inst->completion; + complete(done); + } mutex_unlock(&inst->lock); } } diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2.h index 5fbbab844025..bf3aa1afb5bb 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2.h @@ -20,6 +20,7 @@ struct iris_core; * @packet: HFI packet * @ipsc_properties_set: boolean to set ipsc properties to fw * @opsc_properties_set: boolean to set opsc properties to fw + * @hfi_frame_info: structure of frame info * @src_subcr_params: subscription params to fw on input port * @dst_subcr_params: subscription params to fw on output port */ @@ -28,6 +29,7 @@ struct iris_inst_hfi_gen2 { struct iris_hfi_header *packet; bool ipsc_properties_set; bool opsc_properties_set; + struct iris_hfi_frame_info hfi_frame_info; struct hfi_subscription_params src_subcr_params; struct hfi_subscription_params dst_subcr_params; }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h index d9a903ee1e06..69e383a99d84 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h @@ -70,6 +70,12 @@ #define HFI_SYS_ERROR_WD_TIMEOUT 0x05000001 #define HFI_SYSTEM_ERROR_END 0x05FFFFFF +#define HFI_INFORMATION_BEGIN 0x06000000 +#define HFI_INFO_UNSUPPORTED 0x06000001 +#define HFI_INFO_DATA_CORRUPT 0x06000002 +#define HFI_INFO_BUFFER_OVERFLOW 0x06000004 +#define HFI_INFORMATION_END 0x06FFFFFF + enum hfi_property_mode_type { HFI_MODE_PORT_SETTINGS_CHANGE = 0x00000001, HFI_MODE_PROPERTY = 0x00000002, @@ -91,6 +97,15 @@ enum hfi_codec_type { HFI_CODEC_ENCODE_AVC = 2, }; +enum hfi_picture_type { + HFI_PICTURE_IDR = 0x00000001, + HFI_PICTURE_P = 0x00000002, + HFI_PICTURE_B = 0x00000004, + HFI_PICTURE_I = 0x00000008, + HFI_PICTURE_CRA = 0x00000010, + HFI_PICTURE_BLA = 0x00000020, +}; + enum hfi_buffer_type { HFI_BUFFER_BITSTREAM = 0x00000001, HFI_BUFFER_RAW = 0x00000002, @@ -114,6 +129,13 @@ enum hfi_buffer_host_flags { HFI_BUF_HOST_FLAGS_CB_NON_SECURE = 0x00000200, }; +enum hfi_buffer_firmware_flags { + HFI_BUF_FW_FLAG_RELEASE_DONE = 0x00000001, + HFI_BUF_FW_FLAG_READONLY = 0x00000010, + HFI_BUF_FW_FLAG_LAST = 0x10000000, + HFI_BUF_FW_FLAG_PSC_LAST = 0x20000000, +}; + enum hfi_packet_firmware_flags { HFI_FW_FLAGS_SUCCESS = 0x00000001, HFI_FW_FLAGS_INFORMATION = 0x00000002, diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c index c7f1b6cf102f..1b1dfbd56704 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c @@ -3,6 +3,8 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include + #include "iris_hfi_gen2.h" #include "iris_hfi_gen2_defines.h" #include "iris_hfi_gen2_packet.h" @@ -80,6 +82,29 @@ static bool iris_hfi_gen2_is_valid_hfi_port(u32 port, u32 buffer_type) return true; } +static int iris_hfi_gen2_get_driver_buffer_flags(struct iris_inst *inst, u32 hfi_flags) +{ + u32 keyframe = HFI_PICTURE_IDR | HFI_PICTURE_I | HFI_PICTURE_CRA | HFI_PICTURE_BLA; + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + u32 driver_flags = 0; + + if (inst_hfi_gen2->hfi_frame_info.picture_type & keyframe) + driver_flags |= V4L2_BUF_FLAG_KEYFRAME; + else if (inst_hfi_gen2->hfi_frame_info.picture_type & HFI_PICTURE_P) + driver_flags |= V4L2_BUF_FLAG_PFRAME; + else if (inst_hfi_gen2->hfi_frame_info.picture_type & HFI_PICTURE_B) + driver_flags |= V4L2_BUF_FLAG_BFRAME; + + if (inst_hfi_gen2->hfi_frame_info.data_corrupt || inst_hfi_gen2->hfi_frame_info.overflow) + driver_flags |= V4L2_BUF_FLAG_ERROR; + + if (hfi_flags & HFI_BUF_FW_FLAG_LAST || + hfi_flags & HFI_BUF_FW_FLAG_PSC_LAST) + driver_flags |= V4L2_BUF_FLAG_LAST; + + return driver_flags; +} + static bool iris_hfi_gen2_validate_packet_payload(struct iris_hfi_packet *pkt) { u32 payload_size = 0; @@ -155,6 +180,37 @@ static int iris_hfi_gen2_validate_hdr_packet(struct iris_core *core, struct iris return ret; } +static int iris_hfi_gen2_handle_session_info(struct iris_inst *inst, + struct iris_hfi_packet *pkt) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + struct iris_core *core = inst->core; + int ret = 0; + char *info; + + switch (pkt->type) { + case HFI_INFO_UNSUPPORTED: + info = "unsupported"; + break; + case HFI_INFO_DATA_CORRUPT: + info = "data corrupt"; + inst_hfi_gen2->hfi_frame_info.data_corrupt = 1; + break; + case HFI_INFO_BUFFER_OVERFLOW: + info = "buffer overflow"; + inst_hfi_gen2->hfi_frame_info.overflow = 1; + break; + default: + info = "unknown"; + break; + } + + dev_dbg(core->dev, "session info received %#x: %s\n", + pkt->type, info); + + return ret; +} + static int iris_hfi_gen2_handle_session_error(struct iris_inst *inst, struct iris_hfi_packet *pkt) { @@ -228,6 +284,108 @@ static void iris_hfi_gen2_handle_session_close(struct iris_inst *inst, complete(&inst->completion); } +static int iris_hfi_gen2_handle_input_buffer(struct iris_inst *inst, + struct iris_hfi_buffer *buffer) +{ + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + struct v4l2_m2m_buffer *m2m_buffer, *n; + struct iris_buffer *buf = NULL; + bool found = false; + + v4l2_m2m_for_each_src_buf_safe(m2m_ctx, m2m_buffer, n) { + buf = to_iris_buffer(&m2m_buffer->vb); + if (buf->index == buffer->index) { + found = true; + break; + } + } + if (!found) + return -EINVAL; + + if (!(buf->attr & BUF_ATTR_QUEUED)) + return -EINVAL; + + buf->data_size = buffer->data_size; + buf->attr &= ~BUF_ATTR_QUEUED; + buf->attr |= BUF_ATTR_DEQUEUED; + + buf->flags = iris_hfi_gen2_get_driver_buffer_flags(inst, buffer->flags); + + return 0; +} + +static int iris_hfi_gen2_handle_output_buffer(struct iris_inst *inst, + struct iris_hfi_buffer *hfi_buffer) +{ + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + struct v4l2_m2m_buffer *m2m_buffer, *n; + struct iris_buffer *buf = NULL; + int ret = 0; + bool found = false; + + v4l2_m2m_for_each_dst_buf_safe(m2m_ctx, m2m_buffer, n) { + buf = to_iris_buffer(&m2m_buffer->vb); + if (buf->index == hfi_buffer->index && + buf->device_addr == hfi_buffer->base_address && + buf->data_offset == hfi_buffer->data_offset) { + found = true; + break; + } + } + if (!found) + return -EINVAL; + + if (!(buf->attr & BUF_ATTR_QUEUED)) + return -EINVAL; + + buf->data_offset = hfi_buffer->data_offset; + buf->data_size = hfi_buffer->data_size; + buf->timestamp = hfi_buffer->timestamp; + + buf->attr &= ~BUF_ATTR_QUEUED; + buf->attr |= BUF_ATTR_DEQUEUED; + + buf->flags = iris_hfi_gen2_get_driver_buffer_flags(inst, hfi_buffer->flags); + + return ret; +} + +static int iris_hfi_gen2_handle_dequeue_buffers(struct iris_inst *inst) +{ + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + struct v4l2_m2m_buffer *buffer, *n; + struct iris_buffer *buf = NULL; + int ret = 0; + + v4l2_m2m_for_each_src_buf_safe(m2m_ctx, buffer, n) { + buf = to_iris_buffer(&buffer->vb); + if (buf->attr & BUF_ATTR_DEQUEUED) { + buf->attr &= ~BUF_ATTR_DEQUEUED; + if (!(buf->attr & BUF_ATTR_BUFFER_DONE)) { + buf->attr |= BUF_ATTR_BUFFER_DONE; + ret = iris_vb2_buffer_done(inst, buf); + if (ret) + ret = 0; + } + } + } + + v4l2_m2m_for_each_dst_buf_safe(m2m_ctx, buffer, n) { + buf = to_iris_buffer(&buffer->vb); + if (buf->attr & BUF_ATTR_DEQUEUED) { + buf->attr &= ~BUF_ATTR_DEQUEUED; + if (!(buf->attr & BUF_ATTR_BUFFER_DONE)) { + buf->attr |= BUF_ATTR_BUFFER_DONE; + ret = iris_vb2_buffer_done(inst, buf); + if (ret) + ret = 0; + } + } + } + + return ret; +} + static int iris_hfi_gen2_handle_release_internal_buffer(struct iris_inst *inst, struct iris_hfi_buffer *buffer) { @@ -279,7 +437,12 @@ static int iris_hfi_gen2_handle_session_buffer(struct iris_inst *inst, if (!iris_hfi_gen2_is_valid_hfi_port(pkt->port, buffer->type)) return 0; - return iris_hfi_gen2_handle_release_internal_buffer(inst, buffer); + if (buffer->type == HFI_BUFFER_BITSTREAM) + return iris_hfi_gen2_handle_input_buffer(inst, buffer); + else if (buffer->type == HFI_BUFFER_RAW) + return iris_hfi_gen2_handle_output_buffer(inst, buffer); + else + return iris_hfi_gen2_handle_release_internal_buffer(inst, buffer); } static int iris_hfi_gen2_handle_session_command(struct iris_inst *inst, @@ -342,6 +505,12 @@ static int iris_hfi_gen2_handle_session_property(struct iris_inst *inst, case HFI_PROP_LEVEL: inst_hfi_gen2->src_subcr_params.level = pkt->payload[0]; break; + case HFI_PROP_PICTURE_TYPE: + inst_hfi_gen2->hfi_frame_info.picture_type = pkt->payload[0]; + break; + case HFI_PROP_NO_OUTPUT: + inst_hfi_gen2->hfi_frame_info.no_output = 1; + break; case HFI_PROP_QUALITY_MODE: case HFI_PROP_STAGE: case HFI_PROP_PIPE: @@ -437,14 +606,18 @@ static int iris_hfi_gen2_handle_system_response(struct iris_core *core, static int iris_hfi_gen2_handle_session_response(struct iris_core *core, struct iris_hfi_header *hdr) { + struct iris_inst_hfi_gen2 *inst_hfi_gen2; struct iris_hfi_packet *packet; struct iris_inst *inst; + bool dequeue = false; u8 *pkt, *start_pkt; int ret = 0; int i, j; static const struct iris_hfi_gen2_inst_hfi_range range[] = { {HFI_SESSION_ERROR_BEGIN, HFI_SESSION_ERROR_END, iris_hfi_gen2_handle_session_error}, + {HFI_INFORMATION_BEGIN, HFI_INFORMATION_END, + iris_hfi_gen2_handle_session_info}, {HFI_PROP_BEGIN, HFI_PROP_END, iris_hfi_gen2_handle_session_property}, {HFI_CMD_BEGIN, HFI_CMD_END, @@ -456,6 +629,8 @@ static int iris_hfi_gen2_handle_session_response(struct iris_core *core, return -EINVAL; mutex_lock(&inst->lock); + inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + memset(&inst_hfi_gen2->hfi_frame_info, 0, sizeof(struct iris_hfi_frame_info)); start_pkt = (u8 *)((u8 *)hdr + sizeof(*hdr)); for (i = 0; i < ARRAY_SIZE(range); i++) { @@ -466,6 +641,7 @@ static int iris_hfi_gen2_handle_session_response(struct iris_core *core, iris_hfi_gen2_handle_session_error(inst, packet); if (packet->type > range[i].begin && packet->type < range[i].end) { + dequeue |= (packet->type == HFI_CMD_BUFFER); ret = range[i].handle(inst, packet); if (ret) iris_inst_change_state(inst, IRIS_INST_ERROR); @@ -474,6 +650,13 @@ static int iris_hfi_gen2_handle_session_response(struct iris_core *core, } } + if (dequeue) { + ret = iris_hfi_gen2_handle_dequeue_buffers(inst); + if (ret) + goto unlock; + } + +unlock: mutex_unlock(&inst->lock); return ret; diff --git a/drivers/media/platform/qcom/iris/iris_utils.c b/drivers/media/platform/qcom/iris/iris_utils.c index 7f4e08d392a1..7e84b43d1d4c 100644 --- a/drivers/media/platform/qcom/iris/iris_utils.c +++ b/drivers/media/platform/qcom/iris/iris_utils.c @@ -4,6 +4,7 @@ */ #include +#include #include "iris_instance.h" #include "iris_utils.h" @@ -42,6 +43,21 @@ bool iris_split_mode_enabled(struct iris_inst *inst) return false; } +void iris_helper_buffers_done(struct iris_inst *inst, unsigned int type, + enum vb2_buffer_state state) +{ + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + struct vb2_v4l2_buffer *buf; + + if (V4L2_TYPE_IS_OUTPUT(type)) { + while ((buf = v4l2_m2m_src_buf_remove(m2m_ctx))) + v4l2_m2m_buf_done(buf, state); + } else if (V4L2_TYPE_IS_CAPTURE(type)) { + while ((buf = v4l2_m2m_dst_buf_remove(m2m_ctx))) + v4l2_m2m_buf_done(buf, state); + } +} + int iris_wait_for_session_response(struct iris_inst *inst, bool is_flush) { struct iris_core *core = inst->core; diff --git a/drivers/media/platform/qcom/iris/iris_utils.h b/drivers/media/platform/qcom/iris/iris_utils.h index 0d46f50d3103..5dd7e5ececa5 100644 --- a/drivers/media/platform/qcom/iris/iris_utils.h +++ b/drivers/media/platform/qcom/iris/iris_utils.h @@ -16,6 +16,13 @@ struct iris_hfi_rect_desc { u32 height; }; +struct iris_hfi_frame_info { + u32 picture_type; + u32 no_output; + u32 data_corrupt; + u32 overflow; +}; + #define NUM_MBS_PER_FRAME(height, width) \ (DIV_ROUND_UP(height, 16) * DIV_ROUND_UP(width, 16)) @@ -36,6 +43,8 @@ bool iris_res_is_less_than(u32 width, u32 height, int iris_get_mbpf(struct iris_inst *inst); bool iris_split_mode_enabled(struct iris_inst *inst); struct iris_inst *iris_get_instance(struct iris_core *core, u32 session_id); +void iris_helper_buffers_done(struct iris_inst *inst, unsigned int type, + enum vb2_buffer_state state); int iris_wait_for_session_response(struct iris_inst *inst, bool is_flush); #endif diff --git a/drivers/media/platform/qcom/iris/iris_vb2.c b/drivers/media/platform/qcom/iris/iris_vb2.c index 03b533aef010..391c8225923d 100644 --- a/drivers/media/platform/qcom/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/iris/iris_vb2.c @@ -3,11 +3,24 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include +#include + #include "iris_buffer.h" #include "iris_instance.h" #include "iris_vb2.h" #include "iris_vdec.h" #include "iris_vpu_buffer.h" +int iris_vb2_buf_init(struct vb2_buffer *vb2) +{ + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb2); + struct iris_buffer *buf = NULL; + + buf = to_iris_buffer(vbuf); + buf->device_addr = vb2_dma_contig_plane_dma_addr(vb2, 0); + + return 0; +} int iris_vb2_queue_setup(struct vb2_queue *q, unsigned int *num_buffers, unsigned int *num_planes, @@ -107,6 +120,7 @@ int iris_vb2_queue_setup(struct vb2_queue *q, int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count) { + enum iris_buffer_type buf_type; struct iris_inst *inst; int ret = 0; @@ -136,11 +150,22 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count) if (ret) goto error; + buf_type = iris_v4l2_type_to_driver(q->type); + if (buf_type == -EINVAL) { + ret = -EINVAL; + goto error; + } + + ret = iris_queue_deferred_buffers(inst, buf_type); + if (ret) + goto error; + mutex_unlock(&inst->lock); return ret; error: + iris_helper_buffers_done(inst, q->type, VB2_BUF_STATE_QUEUED); iris_inst_change_state(inst, IRIS_INST_ERROR); mutex_unlock(&inst->lock); @@ -150,6 +175,7 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count) void iris_vb2_stop_streaming(struct vb2_queue *q) { struct iris_inst *inst; + int ret = 0; if (!q) return; @@ -164,8 +190,50 @@ void iris_vb2_stop_streaming(struct vb2_queue *q) !V4L2_TYPE_IS_CAPTURE(q->type)) goto exit; - iris_vdec_session_streamoff(inst, q->type); + ret = iris_vdec_session_streamoff(inst, q->type); + if (ret) + goto exit; + +exit: + iris_helper_buffers_done(inst, q->type, VB2_BUF_STATE_ERROR); + if (ret) + iris_inst_change_state(inst, IRIS_INST_ERROR); + + mutex_unlock(&inst->lock); +} + +void iris_vb2_buf_queue(struct vb2_buffer *vb2) +{ + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb2); + struct v4l2_m2m_ctx *m2m_ctx; + struct iris_inst *inst; + int ret = 0; + + inst = vb2_get_drv_priv(vb2->vb2_queue); + if (!inst) + return; + + mutex_lock(&inst->lock); + if (inst->state == IRIS_INST_ERROR) { + ret = -EBUSY; + goto exit; + } + + m2m_ctx = inst->m2m_ctx; + + if (!vb2->planes[0].bytesused && V4L2_TYPE_IS_OUTPUT(vb2->type)) { + ret = -EINVAL; + goto exit; + } + + v4l2_m2m_buf_queue(m2m_ctx, vbuf); + + ret = iris_vdec_qbuf(inst, vbuf); exit: + if (ret) { + iris_inst_change_state(inst, IRIS_INST_ERROR); + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_ERROR); + } mutex_unlock(&inst->lock); } diff --git a/drivers/media/platform/qcom/iris/iris_vb2.h b/drivers/media/platform/qcom/iris/iris_vb2.h index bc3bb830c2ba..dd3197b2a5bd 100644 --- a/drivers/media/platform/qcom/iris/iris_vb2.h +++ b/drivers/media/platform/qcom/iris/iris_vb2.h @@ -6,10 +6,12 @@ #ifndef _IRIS_VB2_H_ #define _IRIS_VB2_H_ +int iris_vb2_buf_init(struct vb2_buffer *vb2); int iris_vb2_queue_setup(struct vb2_queue *q, unsigned int *num_buffers, unsigned int *num_planes, unsigned int sizes[], struct device *alloc_devs[]); int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count); void iris_vb2_stop_streaming(struct vb2_queue *q); +void iris_vb2_buf_queue(struct vb2_buffer *vb2); #endif diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index d7895fa4b87c..17afb3c123c9 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -256,6 +256,67 @@ int iris_vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_su return ret; } +static int iris_vdec_get_num_queued_buffers(struct iris_inst *inst, + enum iris_buffer_type type) +{ + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + struct v4l2_m2m_buffer *buffer, *n; + struct iris_buffer *buf = NULL; + int count = 0; + + if (type == BUF_INPUT) { + v4l2_m2m_for_each_src_buf_safe(m2m_ctx, buffer, n) { + buf = to_iris_buffer(&buffer->vb); + if (!(buf->attr & BUF_ATTR_QUEUED)) + continue; + count++; + } + } else if (type == BUF_OUTPUT) { + v4l2_m2m_for_each_dst_buf_safe(m2m_ctx, buffer, n) { + buf = to_iris_buffer(&buffer->vb); + if (!(buf->attr & BUF_ATTR_QUEUED)) + continue; + count++; + } + } else { + return count; + } + + return count; +} + +static void iris_vdec_flush_deferred_buffers(struct iris_inst *inst, + enum iris_buffer_type type) +{ + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + struct v4l2_m2m_buffer *buffer, *n; + struct iris_buffer *buf = NULL; + + if (type == BUF_INPUT) { + v4l2_m2m_for_each_src_buf_safe(m2m_ctx, buffer, n) { + buf = to_iris_buffer(&buffer->vb); + if (buf->attr & BUF_ATTR_DEFERRED) { + if (!(buf->attr & BUF_ATTR_BUFFER_DONE)) { + buf->attr |= BUF_ATTR_BUFFER_DONE; + buf->data_size = 0; + iris_vb2_buffer_done(inst, buf); + } + } + } + } else { + v4l2_m2m_for_each_dst_buf_safe(m2m_ctx, buffer, n) { + buf = to_iris_buffer(&buffer->vb); + if (buf->attr & BUF_ATTR_DEFERRED) { + if (!(buf->attr & BUF_ATTR_BUFFER_DONE)) { + buf->attr |= BUF_ATTR_BUFFER_DONE; + buf->data_size = 0; + iris_vb2_buffer_done(inst, buf); + } + } + } + } +} + static void iris_vdec_kill_session(struct iris_inst *inst) { const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; @@ -267,23 +328,43 @@ static void iris_vdec_kill_session(struct iris_inst *inst) iris_inst_change_state(inst, IRIS_INST_ERROR); } -void iris_vdec_session_streamoff(struct iris_inst *inst, u32 plane) +int iris_vdec_session_streamoff(struct iris_inst *inst, u32 plane) { const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + enum iris_buffer_type buffer_type; + int count = 0; int ret; + if (V4L2_TYPE_IS_OUTPUT(plane)) + buffer_type = BUF_INPUT; + else if (V4L2_TYPE_IS_CAPTURE(plane)) + buffer_type = BUF_OUTPUT; + else + return -EINVAL; + ret = hfi_ops->session_stop(inst, plane); if (ret) goto error; + count = iris_vdec_get_num_queued_buffers(inst, buffer_type); + if (count) { + ret = -EINVAL; + goto error; + } + ret = iris_inst_state_change_streamoff(inst, plane); if (ret) goto error; - return; + iris_vdec_flush_deferred_buffers(inst, buffer_type); + + return 0; error: iris_vdec_kill_session(inst); + iris_vdec_flush_deferred_buffers(inst, buffer_type); + + return ret; } static int iris_vdec_process_streamon_input(struct iris_inst *inst) @@ -377,3 +458,47 @@ int iris_vdec_streamon_output(struct iris_inst *inst) return ret; } + +static int +iris_vdec_vb2_buffer_to_driver(struct vb2_buffer *vb2, struct iris_buffer *buf) +{ + struct vb2_v4l2_buffer *vbuf; + u32 buf_type; + + if (!vb2 || !buf) + return -EINVAL; + + vbuf = to_vb2_v4l2_buffer(vb2); + + buf->fd = vb2->planes[0].m.fd; + buf->data_offset = vb2->planes[0].data_offset; + buf->data_size = vb2->planes[0].bytesused - vb2->planes[0].data_offset; + buf->buffer_size = vb2->planes[0].length; + buf->timestamp = vb2->timestamp; + buf->flags = vbuf->flags; + buf->attr = 0; + + buf_type = iris_v4l2_type_to_driver(vb2->type); + if (buf_type == -EINVAL) + return -EINVAL; + + buf->type = iris_v4l2_type_to_driver(vb2->type); + buf->index = vb2->index; + + return 0; +} + +int iris_vdec_qbuf(struct iris_inst *inst, struct vb2_v4l2_buffer *vbuf) +{ + struct vb2_buffer *vb2 = &vbuf->vb2_buf; + struct iris_buffer *buf = NULL; + int ret = 0; + + buf = to_iris_buffer(vbuf); + + ret = iris_vdec_vb2_buffer_to_driver(vb2, buf); + if (ret) + return ret; + + return iris_queue_buffer(inst, buf); +} diff --git a/drivers/media/platform/qcom/iris/iris_vdec.h b/drivers/media/platform/qcom/iris/iris_vdec.h index b3299164f823..51bf0b96d9f0 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.h +++ b/drivers/media/platform/qcom/iris/iris_vdec.h @@ -16,6 +16,7 @@ int iris_vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f); int iris_vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub); int iris_vdec_streamon_input(struct iris_inst *inst); int iris_vdec_streamon_output(struct iris_inst *inst); -void iris_vdec_session_streamoff(struct iris_inst *inst, u32 plane); +int iris_vdec_qbuf(struct iris_inst *inst, struct vb2_v4l2_buffer *vbuf); +int iris_vdec_session_streamoff(struct iris_inst *inst, u32 plane); #endif diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index 1df724588846..34c2548bce9f 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -7,6 +7,7 @@ #include #include #include +#include #include "iris_vidc.h" #include "iris_instance.h" @@ -119,6 +120,7 @@ iris_m2m_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_ src_vq->io_modes = VB2_MMAP | VB2_DMABUF; src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; src_vq->ops = inst->core->iris_vb2_ops; + src_vq->mem_ops = &vb2_dma_contig_memops; src_vq->drv_priv = inst; src_vq->buf_struct_size = sizeof(struct iris_buffer); src_vq->dev = inst->core->dev; @@ -131,6 +133,7 @@ iris_m2m_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_ dst_vq->io_modes = VB2_MMAP | VB2_DMABUF; dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; dst_vq->ops = inst->core->iris_vb2_ops; + dst_vq->mem_ops = &vb2_dma_contig_memops; dst_vq->drv_priv = inst; dst_vq->buf_struct_size = sizeof(struct iris_buffer); dst_vq->dev = inst->core->dev; @@ -610,9 +613,11 @@ static struct v4l2_file_operations iris_v4l2_file_ops = { }; static const struct vb2_ops iris_vb2_ops = { + .buf_init = iris_vb2_buf_init, .queue_setup = iris_vb2_queue_setup, .start_streaming = iris_vb2_start_streaming, .stop_streaming = iris_vb2_stop_streaming, + .buf_queue = iris_vb2_buf_queue, }; static const struct v4l2_ioctl_ops iris_v4l2_ioctl_ops = { @@ -626,6 +631,12 @@ static const struct v4l2_ioctl_ops iris_v4l2_ioctl_ops = { .vidioc_g_fmt_vid_out_mplane = iris_g_fmt_vid_mplane, .vidioc_enum_framesizes = iris_enum_framesizes, .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs, + .vidioc_querybuf = v4l2_m2m_ioctl_querybuf, + .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs, + .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf, + .vidioc_expbuf = v4l2_m2m_ioctl_expbuf, + .vidioc_qbuf = v4l2_m2m_ioctl_qbuf, + .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf, .vidioc_querycap = iris_querycap, .vidioc_queryctrl = iris_queryctrl, .vidioc_querymenu = iris_querymenu, From patchwork Tue Aug 27 10:05:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 824185 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A7D9F1A0AF3; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; cv=none; b=TzgZJ4Ckp/RCw90eyxThzpMuaQ/w+owK/H1ApWGQi6HEWspmDFc3unLytHey1LWwrankrHIwqcBp7s5LKafdBYASMLhN3uTihd1QfLeNjKcPpI7JOsIrTngr5787KG56Wcj9vH0Ugvq5RMAl9uUOIIp8rxFvpz6BH2W8dfPpt4o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; c=relaxed/simple; bh=LLRO//lN3i9LCVuQYFkIuhkWOOEigPCBPLM5YhPCcS8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=gyNZpyrNLlTucZsAE4cCwJq4HCEoy5Wf91rcCwH8/xDBW32HLYC6X1cp5t+U0ozgq2HXc7DTG5c5vGhVmB1NfOIGD1dOkASTATrBHWA+HHlKi/YXqEmliwmXr27ndj4xYwsEljQdd2IE3RLKNb7TSk9Nrl+1l1lr4XAnvL8n/8I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RckJj8Du; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RckJj8Du" Received: by smtp.kernel.org (Postfix) with ESMTPS id 68511C4FE97; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753196; bh=LLRO//lN3i9LCVuQYFkIuhkWOOEigPCBPLM5YhPCcS8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=RckJj8DuvZRi8MU20vwzFgy41k2hXukQ1olFOM7cwkPkRXZ+iAmpGNo6LQWZXALU0 sbTlGhKlLCpQlKkeYido4v33PC91J6x+6cR5tuQRSyignHBjODKUC0XY/0F9X7NDoV V/kHvMy4FvTlkEgHane5nd3pxz2la9Z7Jkh4UQalGnx53DNdLGdYuac+3J6783Vs+T +WGUewyN4RuwoAajDmkPYZZ9MC4ccwHxkOch6NkfqaJ/zW0UtFS7vcReYexbNdneyD b1vm+eXliGylWiiQPa19AubXt71QZf94rblrTJrKqcIeznxhHiTf+Cw2zpsfpC3Yc4 /RfK4nLARDc/w== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5C7A5C54736; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:48 +0530 Subject: [PATCH v3 23/29] media: iris: add support for dynamic resolution change Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-23-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=34685; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=upBLZDwdKpnMwlbB2QnyIaK5IaJu314T3xLvE9tO2p8=; b=gjXWZqhK+9CNDHVL74c4JHkMTdV/F6CJ319Wyq9C7MCffJDjXwtInmDPD3cZS90ADRaSEeDY8 B8eewcp8PfaA5i71pUq9z2R3Jpc9tsUUhKP/9hhDJZCqXccTflsh2i6 X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal Handle the response sent by firmware, when source change is detected by firmware. Read the parameter subscribed to firmware for source change to detect the parameter with changed value. Raise the source change event to client and update the instance sub state. Mark the last buffer from before the source change with the V4L2_BUF_FLAG_LAST flag and return to client. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/iris_hfi_common.c | 96 ++++++++++ drivers/media/platform/qcom/iris/iris_hfi_common.h | 3 + .../platform/qcom/iris/iris_hfi_gen1_defines.h | 82 +++++++++ .../platform/qcom/iris/iris_hfi_gen1_response.c | 204 +++++++++++++++++++++ .../platform/qcom/iris/iris_hfi_gen2_defines.h | 4 + .../platform/qcom/iris/iris_hfi_gen2_response.c | 179 ++++++++++++++++++ drivers/media/platform/qcom/iris/iris_instance.h | 2 + drivers/media/platform/qcom/iris/iris_state.c | 64 +++++++ drivers/media/platform/qcom/iris/iris_state.h | 35 ++++ drivers/media/platform/qcom/iris/iris_vb2.c | 15 ++ drivers/media/platform/qcom/iris/iris_vdec.c | 15 ++ drivers/media/platform/qcom/iris/iris_vdec.h | 1 + 12 files changed, 700 insertions(+) diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.c b/drivers/media/platform/qcom/iris/iris_hfi_common.c index bf9ec678ff90..e1a21ea73ad9 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_common.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_common.c @@ -11,6 +11,102 @@ #include "iris_power.h" #include "iris_vpu_common.h" +u32 iris_hfi_get_v4l2_color_primaries(u32 hfi_primaries) +{ + u32 primaries = V4L2_COLORSPACE_DEFAULT; + + switch (hfi_primaries) { + case HFI_PRIMARIES_RESERVED: + primaries = V4L2_COLORSPACE_DEFAULT; + break; + case HFI_PRIMARIES_BT709: + primaries = V4L2_COLORSPACE_REC709; + break; + case HFI_PRIMARIES_BT470_SYSTEM_M: + primaries = V4L2_COLORSPACE_470_SYSTEM_M; + break; + case HFI_PRIMARIES_BT470_SYSTEM_BG: + primaries = V4L2_COLORSPACE_470_SYSTEM_BG; + break; + case HFI_PRIMARIES_BT601_525: + primaries = V4L2_COLORSPACE_SMPTE170M; + break; + case HFI_PRIMARIES_SMPTE_ST240M: + primaries = V4L2_COLORSPACE_SMPTE240M; + break; + case HFI_PRIMARIES_BT2020: + primaries = V4L2_COLORSPACE_BT2020; + break; + case V4L2_COLORSPACE_DCI_P3: + primaries = HFI_PRIMARIES_SMPTE_RP431_2; + break; + default: + break; + } + + return primaries; +} + +u32 iris_hfi_get_v4l2_transfer_char(u32 hfi_characterstics) +{ + u32 characterstics = V4L2_XFER_FUNC_DEFAULT; + + switch (hfi_characterstics) { + case HFI_TRANSFER_RESERVED: + characterstics = V4L2_XFER_FUNC_DEFAULT; + break; + case HFI_TRANSFER_BT709: + characterstics = V4L2_XFER_FUNC_709; + break; + case HFI_TRANSFER_SMPTE_ST240M: + characterstics = V4L2_XFER_FUNC_SMPTE240M; + break; + case HFI_TRANSFER_SRGB_SYCC: + characterstics = V4L2_XFER_FUNC_SRGB; + break; + case HFI_TRANSFER_SMPTE_ST2084_PQ: + characterstics = V4L2_XFER_FUNC_SMPTE2084; + break; + default: + break; + } + + return characterstics; +} + +u32 iris_hfi_get_v4l2_matrix_coefficients(u32 hfi_coefficients) +{ + u32 coefficients = V4L2_YCBCR_ENC_DEFAULT; + + switch (hfi_coefficients) { + case HFI_MATRIX_COEFF_RESERVED: + coefficients = V4L2_YCBCR_ENC_DEFAULT; + break; + case HFI_MATRIX_COEFF_BT709: + coefficients = V4L2_YCBCR_ENC_709; + break; + case HFI_MATRIX_COEFF_BT470_SYS_BG_OR_BT601_625: + coefficients = V4L2_YCBCR_ENC_XV601; + break; + case HFI_MATRIX_COEFF_BT601_525_BT1358_525_OR_625: + coefficients = V4L2_YCBCR_ENC_601; + break; + case HFI_MATRIX_COEFF_SMPTE_ST240: + coefficients = V4L2_YCBCR_ENC_SMPTE240M; + break; + case HFI_MATRIX_COEFF_BT2020_NON_CONSTANT: + coefficients = V4L2_YCBCR_ENC_BT2020; + break; + case HFI_MATRIX_COEFF_BT2020_CONSTANT: + coefficients = V4L2_YCBCR_ENC_BT2020_CONST_LUM; + break; + default: + break; + } + + return coefficients; +} + int iris_hfi_core_init(struct iris_core *core) { const struct iris_hfi_command_ops *hfi_ops = core->hfi_ops; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.h b/drivers/media/platform/qcom/iris/iris_hfi_common.h index 9a6bb72e6134..22b0cbfcce76 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_common.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_common.h @@ -138,6 +138,9 @@ struct hfi_subscription_params { u32 level; }; +u32 iris_hfi_get_v4l2_color_primaries(u32 hfi_primaries); +u32 iris_hfi_get_v4l2_transfer_char(u32 hfi_characterstics); +u32 iris_hfi_get_v4l2_matrix_coefficients(u32 hfi_coefficients); int iris_hfi_core_init(struct iris_core *core); int iris_hfi_pm_suspend(struct iris_core *core); int iris_hfi_pm_resume(struct iris_core *core); diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h index b7f9e0f47675..41e103744bd5 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h @@ -44,18 +44,28 @@ #define HFI_EVENT_SYS_ERROR 0x1 #define HFI_EVENT_SESSION_ERROR 0x2 +#define HFI_EVENT_DATA_SEQUENCE_CHANGED_SUFFICIENT_BUF_RESOURCES 0x1000001 +#define HFI_EVENT_DATA_SEQUENCE_CHANGED_INSUFFICIENT_BUF_RESOURCES 0x1000002 +#define HFI_EVENT_SESSION_SEQUENCE_CHANGED 0x1000003 + #define HFI_BUFFERFLAG_TIMESTAMPINVALID 0x00000100 #define HFI_FLUSH_OUTPUT 0x1000002 #define HFI_FLUSH_OUTPUT2 0x1000003 #define HFI_FLUSH_ALL 0x1000004 +#define HFI_INDEX_EXTRADATA_INPUT_CROP 0x0700000e + #define HFI_PROPERTY_PARAM_BUFFER_COUNT_ACTUAL 0x201001 #define HFI_PROPERTY_PARAM_UNCOMPRESSED_PLANE_ACTUAL_CONSTRAINTS_INFO 0x201002 #define HFI_PROPERTY_PARAM_BUFFER_ALLOC_MODE 0x201008 #define HFI_PROPERTY_PARAM_BUFFER_SIZE_ACTUAL 0x20100c +#define HFI_PROPERTY_CONFIG_BUFFER_REQUIREMENTS 0x202001 + #define HFI_PROPERTY_CONFIG_VDEC_POST_LOOP_DEBLOCKER 0x1200001 +#define HFI_PROPERTY_PARAM_VDEC_DPB_COUNTS 0x120300e +#define HFI_PROPERTY_CONFIG_VDEC_ENTROPY 0x1204004 #define HFI_BUFFER_INPUT 0x1 #define HFI_BUFFER_OUTPUT 0x2 @@ -69,11 +79,15 @@ #define HFI_PROPERTY_PARAM_FRAME_SIZE 0x1001 #define HFI_PROPERTY_PARAM_UNCOMPRESSED_FORMAT_SELECT 0x1003 +#define HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT 0x1005 #define HFI_PROPERTY_PARAM_WORK_MODE 0x1015 #define HFI_PROPERTY_PARAM_WORK_ROUTE 0x1017 #define HFI_PROPERTY_CONFIG_VIDEOCORES_USAGE 0x2002 #define HFI_PROPERTY_PARAM_VDEC_MULTI_STREAM 0x1003001 +#define HFI_PROPERTY_PARAM_VDEC_PIXEL_BITDEPTH 0x1003007 +#define HFI_PROPERTY_PARAM_VDEC_PIC_STRUCT 0x1003009 +#define HFI_PROPERTY_PARAM_VDEC_COLOUR_SPACE 0x100300a #define HFI_CORE_ID_1 1 #define HFI_COLOR_FORMAT_NV12 0x02 #define HFI_COLOR_FORMAT_NV12_UBWC 0x8002 @@ -249,6 +263,11 @@ struct hfi_enable { u32 enable; }; +struct hfi_profile_level { + u32 profile; + u32 level; +}; + struct hfi_framesize { u32 buffer_type; u32 width; @@ -267,6 +286,37 @@ struct hfi_video_work_route { u32 video_work_route; }; +struct hfi_bit_depth { + u32 buffer_type; + u32 bit_depth; +}; + +struct hfi_pic_struct { + u32 progressive_only; +}; + +struct hfi_colour_space { + u32 colour_space; +}; + +struct hfi_extradata_input_crop { + u32 size; + u32 version; + u32 port_index; + u32 left; + u32 top; + u32 width; + u32 height; +}; + +struct hfi_dpb_counts { + u32 max_dpb_count; + u32 max_ref_frames; + u32 max_dec_buffering; + u32 max_reorder_frames; + u32 fw_min_count; +}; + struct hfi_uncompressed_format_select { u32 buffer_type; u32 format; @@ -301,6 +351,38 @@ struct hfi_multi_stream { u32 enable; }; +struct hfi_buffer_requirements { + u32 type; + u32 size; + u32 region_size; + u32 hold_count; + u32 count_min; + u32 count_actual; + u32 contiguous; + u32 alignment; +}; + +struct hfi_event_data { + u32 error; + u32 height; + u32 width; + u32 event_type; + u32 packet_buffer; + u32 extradata_buffer; + u32 tag; + u32 profile; + u32 level; + u32 bit_depth; + u32 pic_struct; + u32 colour_space; + u32 entropy_mode; + u32 buf_count; + struct { + u32 left, top; + u32 width, height; + } input_crop; +}; + struct hfi_msg_session_empty_buffer_done_pkt { struct hfi_msg_session_hdr_pkt shdr; u32 offset; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c index 32fb01616122..e3d9bfb8bef6 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c @@ -3,11 +3,212 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ +#include #include #include "iris_hfi_gen1.h" #include "iris_hfi_gen1_defines.h" #include "iris_instance.h" +#include "iris_vdec.h" +#include "iris_vpu_buffer.h" + +static void iris_hfi_gen1_read_changed_params(struct iris_inst *inst, + struct hfi_msg_event_notify_pkt *pkt) +{ + struct v4l2_pix_format_mplane *pixmp_ip, *pixmp_op; + u32 primaries, matrix_coeff, transfer_char; + struct hfi_dpb_counts *iris_vpu_dpb_count; + struct hfi_profile_level *profile_level; + u32 colour_description_present_flag = 0; + struct hfi_buffer_requirements *bufreq; + u32 video_signal_type_present_flag = 0; + struct hfi_extradata_input_crop *crop; + struct hfi_colour_space *colour_info; + struct iris_core *core = inst->core; + struct hfi_event_data event = {0}; + struct hfi_bit_depth *pixel_depth; + struct hfi_pic_struct *pic_struct; + struct hfi_framesize *frame_sz; + int num_properties_changed; + u32 full_range = 0; + u8 *data_ptr; + u32 ptype; + + pixmp_ip = &inst->fmt_src->fmt.pix_mp; + pixmp_op = &inst->fmt_dst->fmt.pix_mp; + num_properties_changed = pkt->event_data2; + + data_ptr = (u8 *)&pkt->ext_event_data[0]; + do { + ptype = *((u32 *)data_ptr); + switch (ptype) { + case HFI_PROPERTY_PARAM_FRAME_SIZE: + data_ptr += sizeof(u32); + frame_sz = (struct hfi_framesize *)data_ptr; + event.width = frame_sz->width; + event.height = frame_sz->height; + data_ptr += sizeof(*frame_sz); + break; + case HFI_PROPERTY_PARAM_PROFILE_LEVEL_CURRENT: + data_ptr += sizeof(u32); + profile_level = (struct hfi_profile_level *)data_ptr; + event.profile = profile_level->profile; + event.level = profile_level->level; + data_ptr += sizeof(*profile_level); + break; + case HFI_PROPERTY_PARAM_VDEC_PIXEL_BITDEPTH: + data_ptr += sizeof(u32); + pixel_depth = (struct hfi_bit_depth *)data_ptr; + event.bit_depth = pixel_depth->bit_depth; + data_ptr += sizeof(*pixel_depth); + break; + case HFI_PROPERTY_PARAM_VDEC_PIC_STRUCT: + data_ptr += sizeof(u32); + pic_struct = (struct hfi_pic_struct *)data_ptr; + event.pic_struct = pic_struct->progressive_only; + data_ptr += sizeof(*pic_struct); + break; + case HFI_PROPERTY_PARAM_VDEC_COLOUR_SPACE: + data_ptr += sizeof(u32); + colour_info = (struct hfi_colour_space *)data_ptr; + event.colour_space = colour_info->colour_space; + data_ptr += sizeof(*colour_info); + break; + case HFI_PROPERTY_CONFIG_VDEC_ENTROPY: + data_ptr += sizeof(u32); + event.entropy_mode = *(u32 *)data_ptr; + data_ptr += sizeof(u32); + break; + case HFI_PROPERTY_CONFIG_BUFFER_REQUIREMENTS: + data_ptr += sizeof(u32); + bufreq = (struct hfi_buffer_requirements *)data_ptr; + event.buf_count = bufreq->count_min; + data_ptr += sizeof(*bufreq); + break; + case HFI_INDEX_EXTRADATA_INPUT_CROP: + data_ptr += sizeof(u32); + crop = (struct hfi_extradata_input_crop *)data_ptr; + event.input_crop.left = crop->left; + event.input_crop.top = crop->top; + event.input_crop.width = crop->width; + event.input_crop.height = crop->height; + data_ptr += sizeof(*crop); + break; + case HFI_PROPERTY_PARAM_VDEC_DPB_COUNTS: + data_ptr += sizeof(u32); + iris_vpu_dpb_count = (struct hfi_dpb_counts *)data_ptr; + event.buf_count = iris_vpu_dpb_count->fw_min_count; + data_ptr += sizeof(*iris_vpu_dpb_count); + break; + default: + break; + } + num_properties_changed--; + } while (num_properties_changed > 0); + + pixmp_ip->width = event.width; + pixmp_ip->height = event.height; + + pixmp_op->width = ALIGN(event.width, 128); + pixmp_op->height = ALIGN(event.height, 32); + pixmp_op->plane_fmt[0].bytesperline = ALIGN(event.width, 128); + pixmp_op->plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT); + + matrix_coeff = FIELD_GET(GENMASK(7, 0), event.colour_space); + transfer_char = FIELD_GET(GENMASK(15, 8), event.colour_space); + primaries = FIELD_GET(GENMASK(23, 16), event.colour_space); + colour_description_present_flag = FIELD_GET(GENMASK(24, 24), event.colour_space); + full_range = FIELD_GET(GENMASK(25, 25), event.colour_space); + video_signal_type_present_flag = FIELD_GET(GENMASK(29, 29), event.colour_space); + + pixmp_op->colorspace = V4L2_COLORSPACE_DEFAULT; + pixmp_op->xfer_func = V4L2_XFER_FUNC_DEFAULT; + pixmp_op->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + pixmp_op->quantization = V4L2_QUANTIZATION_DEFAULT; + + if (video_signal_type_present_flag) { + pixmp_op->quantization = + full_range ? + V4L2_QUANTIZATION_FULL_RANGE : + V4L2_QUANTIZATION_LIM_RANGE; + if (colour_description_present_flag) { + pixmp_op->colorspace = + iris_hfi_get_v4l2_color_primaries(primaries); + pixmp_op->xfer_func = + iris_hfi_get_v4l2_transfer_char(transfer_char); + pixmp_op->ycbcr_enc = + iris_hfi_get_v4l2_matrix_coefficients(matrix_coeff); + } + } + + pixmp_ip->colorspace = pixmp_op->colorspace; + pixmp_ip->xfer_func = pixmp_op->xfer_func; + pixmp_ip->ycbcr_enc = pixmp_op->ycbcr_enc; + pixmp_ip->quantization = pixmp_op->quantization; + + if (event.input_crop.width > 0 && event.input_crop.height > 0) { + inst->crop.left = event.input_crop.left; + inst->crop.top = event.input_crop.top; + inst->crop.width = event.input_crop.width; + inst->crop.height = event.input_crop.height; + } else { + inst->crop.left = 0; + inst->crop.top = 0; + inst->crop.width = event.width; + inst->crop.height = event.height; + } + + inst->fw_min_count = event.buf_count; + inst->buffers[BUF_OUTPUT].min_count = iris_vpu_buf_count(inst, BUF_OUTPUT); + + if (event.bit_depth || !event.pic_struct) { + dev_err(core->dev, "unsupported content, bit depth: %x, pic_struct = %x\n", + event.bit_depth, event.pic_struct); + iris_inst_change_state(inst, IRIS_INST_ERROR); + } +} + +static void iris_hfi_gen1_event_seq_changed(struct iris_inst *inst, + struct hfi_msg_event_notify_pkt *pkt) +{ + struct hfi_session_flush_pkt flush_pkt; + int num_properties_changed; + int ret; + + ret = iris_inst_sub_state_change_drc(inst); + if (ret) + return; + + switch (pkt->event_data1) { + case HFI_EVENT_DATA_SEQUENCE_CHANGED_SUFFICIENT_BUF_RESOURCES: + case HFI_EVENT_DATA_SEQUENCE_CHANGED_INSUFFICIENT_BUF_RESOURCES: + break; + default: + iris_inst_change_state(inst, IRIS_INST_ERROR); + return; + } + + num_properties_changed = pkt->event_data2; + if (!num_properties_changed) { + iris_inst_change_state(inst, IRIS_INST_ERROR); + return; + } + + iris_hfi_gen1_read_changed_params(inst, pkt); + + if (inst->state != IRIS_INST_ERROR) { + reinit_completion(&inst->flush_completion); + + flush_pkt.shdr.hdr.size = sizeof(struct hfi_session_flush_pkt); + flush_pkt.shdr.hdr.pkt_type = HFI_CMD_SESSION_FLUSH; + flush_pkt.shdr.session_id = inst->session_id; + flush_pkt.flush_type = HFI_FLUSH_OUTPUT; + iris_hfi_queue_cmd_write(inst->core, &flush_pkt, flush_pkt.shdr.hdr.size); + } + + iris_vdec_src_change(inst); + iris_inst_sub_state_change_drc_last(inst); +} static void iris_hfi_gen1_sys_event_notify(struct iris_core *core, void *packet) @@ -59,6 +260,9 @@ static void iris_hfi_gen1_session_event_notify(struct iris_inst *inst, void *pac case HFI_EVENT_SESSION_ERROR: iris_hfi_gen1_event_session_error(inst, pkt); break; + case HFI_EVENT_SESSION_SEQUENCE_CHANGED: + iris_hfi_gen1_event_seq_changed(inst, pkt); + break; default: break; } diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h index 69e383a99d84..66ffce02ae57 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h @@ -19,8 +19,11 @@ #define HFI_CMD_STOP 0x01000006 #define HFI_CMD_BUFFER 0x01000009 #define HFI_CMD_SUBSCRIBE_MODE 0x0100000B +#define HFI_CMD_SETTINGS_CHANGE 0x0100000C #define HFI_CMD_END 0x01FFFFFF +#define HFI_BITMASK_BITSTREAM_WIDTH 0xffff0000 +#define HFI_BITMASK_BITSTREAM_HEIGHT 0x0000ffff #define HFI_BITMASK_FRAME_MBS_ONLY_FLAG 0x00000001 #define HFI_PROP_BEGIN 0x03000000 @@ -74,6 +77,7 @@ #define HFI_INFO_UNSUPPORTED 0x06000001 #define HFI_INFO_DATA_CORRUPT 0x06000002 #define HFI_INFO_BUFFER_OVERFLOW 0x06000004 +#define HFI_INFO_HFI_FLAG_PSC_LAST 0x06000007 #define HFI_INFORMATION_END 0x06FFFFFF enum hfi_property_mode_type { diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c index 1b1dfbd56704..626a858e803c 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c @@ -8,6 +8,8 @@ #include "iris_hfi_gen2.h" #include "iris_hfi_gen2_defines.h" #include "iris_hfi_gen2_packet.h" +#include "iris_vdec.h" +#include "iris_vpu_buffer.h" #include "iris_vpu_common.h" struct iris_hfi_gen2_core_hfi_range { @@ -200,6 +202,10 @@ static int iris_hfi_gen2_handle_session_info(struct iris_inst *inst, info = "buffer overflow"; inst_hfi_gen2->hfi_frame_info.overflow = 1; break; + case HFI_INFO_HFI_FLAG_PSC_LAST: + info = "drc last flag"; + ret = iris_inst_sub_state_change_drc_last(inst); + break; default: info = "unknown"; break; @@ -323,6 +329,12 @@ static int iris_hfi_gen2_handle_output_buffer(struct iris_inst *inst, int ret = 0; bool found = false; + if (hfi_buffer->flags & HFI_BUF_FW_FLAG_PSC_LAST) { + ret = iris_inst_sub_state_change_drc_last(inst); + if (ret) + return ret; + } + v4l2_m2m_for_each_dst_buf_safe(m2m_ctx, m2m_buffer, n) { buf = to_iris_buffer(&m2m_buffer->vb); if (buf->index == hfi_buffer->index && @@ -445,6 +457,108 @@ static int iris_hfi_gen2_handle_session_buffer(struct iris_inst *inst, return iris_hfi_gen2_handle_release_internal_buffer(inst, buffer); } +static void iris_hfi_gen2_read_input_subcr_params(struct iris_inst *inst) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + struct v4l2_pix_format_mplane *pixmp_ip, *pixmp_op; + u32 primaries, matrix_coeff, transfer_char; + struct hfi_subscription_params subsc_params; + u32 colour_description_present_flag = 0; + u32 video_signal_type_present_flag = 0; + struct iris_core *core = inst->core; + u32 full_range = 0; + u32 width, height; + + subsc_params = inst_hfi_gen2->src_subcr_params; + pixmp_ip = &inst->fmt_src->fmt.pix_mp; + pixmp_op = &inst->fmt_dst->fmt.pix_mp; + width = (subsc_params.bitstream_resolution & + HFI_BITMASK_BITSTREAM_WIDTH) >> 16; + height = subsc_params.bitstream_resolution & + HFI_BITMASK_BITSTREAM_HEIGHT; + + pixmp_ip->width = width; + pixmp_ip->height = height; + + pixmp_op->width = ALIGN(width, 128); + pixmp_op->height = ALIGN(height, 32); + pixmp_op->plane_fmt[0].bytesperline = ALIGN(width, 128); + pixmp_op->plane_fmt[0].sizeimage = iris_get_buffer_size(inst, BUF_OUTPUT); + + matrix_coeff = subsc_params.color_info & 0xFF; + transfer_char = (subsc_params.color_info & 0xFF00) >> 8; + primaries = (subsc_params.color_info & 0xFF0000) >> 16; + colour_description_present_flag = + (subsc_params.color_info & 0x1000000) >> 24; + full_range = (subsc_params.color_info & 0x2000000) >> 25; + video_signal_type_present_flag = + (subsc_params.color_info & 0x20000000) >> 29; + + pixmp_op->colorspace = V4L2_COLORSPACE_DEFAULT; + pixmp_op->xfer_func = V4L2_XFER_FUNC_DEFAULT; + pixmp_op->ycbcr_enc = V4L2_YCBCR_ENC_DEFAULT; + pixmp_op->quantization = V4L2_QUANTIZATION_DEFAULT; + + if (video_signal_type_present_flag) { + pixmp_op->quantization = + full_range ? + V4L2_QUANTIZATION_FULL_RANGE : + V4L2_QUANTIZATION_LIM_RANGE; + if (colour_description_present_flag) { + pixmp_op->colorspace = + iris_hfi_get_v4l2_color_primaries(primaries); + pixmp_op->xfer_func = + iris_hfi_get_v4l2_transfer_char(transfer_char); + pixmp_op->ycbcr_enc = + iris_hfi_get_v4l2_matrix_coefficients(matrix_coeff); + } + } + + pixmp_ip->colorspace = pixmp_op->colorspace; + pixmp_ip->xfer_func = pixmp_op->xfer_func; + pixmp_ip->ycbcr_enc = pixmp_op->ycbcr_enc; + pixmp_ip->quantization = pixmp_op->quantization; + + inst->crop.top = subsc_params.crop_offsets[0] & 0xFFFF; + inst->crop.left = (subsc_params.crop_offsets[0] >> 16) & 0xFFFF; + inst->crop.height = pixmp_ip->height - + (subsc_params.crop_offsets[1] & 0xFFFF) - inst->crop.top; + inst->crop.width = pixmp_ip->width - + ((subsc_params.crop_offsets[1] >> 16) & 0xFFFF) - inst->crop.left; + + inst->fw_cap[PROFILE].value = subsc_params.profile; + inst->fw_cap[LEVEL].value = subsc_params.level; + inst->fw_cap[POC].value = subsc_params.pic_order_cnt; + + if (subsc_params.bit_depth != BIT_DEPTH_8 || + !(subsc_params.coded_frames & HFI_BITMASK_FRAME_MBS_ONLY_FLAG)) { + dev_err(core->dev, "unsupported content, bit depth: %x, pic_struct = %x\n", + subsc_params.bit_depth, subsc_params.coded_frames); + iris_inst_change_state(inst, IRIS_INST_ERROR); + } + + inst->fw_min_count = subsc_params.fw_min_count; + inst->buffers[BUF_OUTPUT].min_count = iris_vpu_buf_count(inst, BUF_OUTPUT); +} + +static int iris_hfi_gen2_handle_src_change(struct iris_inst *inst, + struct iris_hfi_packet *pkt) +{ + int ret; + + if (pkt->port != HFI_PORT_BITSTREAM) + return 0; + + ret = iris_inst_sub_state_change_drc(inst); + if (ret) + return ret; + + iris_hfi_gen2_read_input_subcr_params(inst); + iris_vdec_src_change(inst); + + return 0; +} + static int iris_hfi_gen2_handle_session_command(struct iris_inst *inst, struct iris_hfi_packet *pkt) { @@ -460,6 +574,9 @@ static int iris_hfi_gen2_handle_session_command(struct iris_inst *inst, case HFI_CMD_BUFFER: ret = iris_hfi_gen2_handle_session_buffer(inst, pkt); break; + case HFI_CMD_SETTINGS_CHANGE: + ret = iris_hfi_gen2_handle_src_change(inst, pkt); + break; default: break; } @@ -603,6 +720,56 @@ static int iris_hfi_gen2_handle_system_response(struct iris_core *core, return ret; } +static void iris_hfi_gen2_init_src_change_param(struct iris_inst *inst) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + u32 left_offset, top_offset, right_offset, bottom_offset; + struct v4l2_pix_format_mplane *pixmp_ip, *pixmp_op; + u32 primaries, matrix_coeff, transfer_char; + struct hfi_subscription_params *subsc_params; + u32 colour_description_present_flag = 0; + u32 video_signal_type_present_flag = 0; + u32 full_range = 0, video_format = 0; + + subsc_params = &inst_hfi_gen2->src_subcr_params; + pixmp_ip = &inst->fmt_src->fmt.pix_mp; + pixmp_op = &inst->fmt_dst->fmt.pix_mp; + + subsc_params->bitstream_resolution = + pixmp_ip->width << 16 | pixmp_ip->height; + + left_offset = inst->crop.left; + top_offset = inst->crop.top; + right_offset = (pixmp_ip->width - inst->crop.width); + bottom_offset = (pixmp_ip->height - inst->crop.height); + subsc_params->crop_offsets[0] = + left_offset << 16 | top_offset; + subsc_params->crop_offsets[1] = + right_offset << 16 | bottom_offset; + + subsc_params->fw_min_count = inst->buffers[BUF_OUTPUT].min_count; + + primaries = iris_hfi_gen2_get_color_primaries(pixmp_op->colorspace); + matrix_coeff = iris_hfi_gen2_get_matrix_coefficients(pixmp_op->ycbcr_enc); + transfer_char = iris_hfi_gen2_get_transfer_char(pixmp_op->xfer_func); + full_range = pixmp_op->quantization == V4L2_QUANTIZATION_FULL_RANGE ? 1 : 0; + subsc_params->color_info = + iris_hfi_gen2_get_color_info(matrix_coeff, transfer_char, primaries, + colour_description_present_flag, + full_range, video_format, + video_signal_type_present_flag); + + subsc_params->profile = inst->fw_cap[PROFILE].value; + subsc_params->level = inst->fw_cap[LEVEL].value; + subsc_params->pic_order_cnt = inst->fw_cap[POC].value; + subsc_params->bit_depth = inst->fw_cap[BIT_DEPTH].value; + if (inst->fw_cap[CODED_FRAMES].value == + CODED_FRAMES_PROGRESSIVE) + subsc_params->coded_frames = HFI_BITMASK_FRAME_MBS_ONLY_FLAG; + else + subsc_params->coded_frames = 0; +} + static int iris_hfi_gen2_handle_session_response(struct iris_core *core, struct iris_hfi_header *hdr) { @@ -632,6 +799,18 @@ static int iris_hfi_gen2_handle_session_response(struct iris_core *core, inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); memset(&inst_hfi_gen2->hfi_frame_info, 0, sizeof(struct iris_hfi_frame_info)); + pkt = (u8 *)((u8 *)hdr + sizeof(*hdr)); + for (i = 0; i < hdr->num_packets; i++) { + packet = (struct iris_hfi_packet *)pkt; + if (packet->type == HFI_CMD_SETTINGS_CHANGE) { + if (packet->port == HFI_PORT_BITSTREAM) { + iris_hfi_gen2_init_src_change_param(inst); + break; + } + } + pkt += packet->size; + } + start_pkt = (u8 *)((u8 *)hdr + sizeof(*hdr)); for (i = 0; i < ARRAY_SIZE(range); i++) { pkt = start_pkt; diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h index c225466b4e48..dd1dad586a69 100644 --- a/drivers/media/platform/qcom/iris/iris_instance.h +++ b/drivers/media/platform/qcom/iris/iris_instance.h @@ -32,6 +32,7 @@ * @buffers: array of different iris buffers * @fw_min_count: minimnum count of buffers needed by fw * @state: instance state + * @sub_state: instance sub state * @once_per_session_set: boolean to set once per session property * @m2m_dev: a reference to m2m device structure * @m2m_ctx: a reference to m2m context structure @@ -56,6 +57,7 @@ struct iris_inst { struct iris_buffers buffers[BUF_TYPE_MAX]; u32 fw_min_count; enum iris_inst_state state; + enum iris_inst_sub_state sub_state; bool once_per_session_set; struct v4l2_m2m_dev *m2m_dev; struct v4l2_m2m_ctx *m2m_ctx; diff --git a/drivers/media/platform/qcom/iris/iris_state.c b/drivers/media/platform/qcom/iris/iris_state.c index 14a7f03a012c..e39c110b75e0 100644 --- a/drivers/media/platform/qcom/iris/iris_state.c +++ b/drivers/media/platform/qcom/iris/iris_state.c @@ -114,3 +114,67 @@ int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane) return iris_inst_change_state(inst, new_state); } + +int iris_inst_change_sub_state(struct iris_inst *inst, + enum iris_inst_sub_state clear_sub_state, + enum iris_inst_sub_state set_sub_state) +{ + enum iris_inst_sub_state prev_sub_state; + + if (inst->state == IRIS_INST_ERROR) + return 0; + + if (!clear_sub_state && !set_sub_state) + return 0; + + if ((clear_sub_state & set_sub_state) || + set_sub_state > IRIS_INST_MAX_SUB_STATE_VALUE || + clear_sub_state > IRIS_INST_MAX_SUB_STATE_VALUE) + return -EINVAL; + + prev_sub_state = inst->sub_state; + + inst->sub_state |= set_sub_state; + inst->sub_state &= ~clear_sub_state; + + if (inst->sub_state != prev_sub_state) + dev_dbg(inst->core->dev, "sub_state changed from %x to %x\n", + prev_sub_state, inst->sub_state); + + return 0; +} + +int iris_inst_sub_state_change_drc(struct iris_inst *inst) +{ + enum iris_inst_sub_state set_sub_state = 0; + + if (inst->sub_state & IRIS_INST_SUB_DRC) + return -EINVAL; + + if (inst->state == IRIS_INST_INPUT_STREAMING || + inst->state == IRIS_INST_INIT) + set_sub_state = IRIS_INST_SUB_FIRST_IPSC | IRIS_INST_SUB_INPUT_PAUSE; + else + set_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_INPUT_PAUSE; + + return iris_inst_change_sub_state(inst, 0, set_sub_state); +} + +int iris_inst_sub_state_change_drc_last(struct iris_inst *inst) +{ + enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE; + + if (inst->sub_state & IRIS_INST_SUB_DRC_LAST) + return -EINVAL; + + if (!(inst->sub_state & IRIS_INST_SUB_DRC) || + !(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE)) + return -EINVAL; + + if (inst->sub_state & IRIS_INST_SUB_FIRST_IPSC) + return 0; + + set_sub_state = IRIS_INST_SUB_DRC_LAST | IRIS_INST_SUB_OUTPUT_PAUSE; + + return iris_inst_change_sub_state(inst, 0, set_sub_state); +} diff --git a/drivers/media/platform/qcom/iris/iris_state.h b/drivers/media/platform/qcom/iris/iris_state.h index d6613970a24e..786b82ca0307 100644 --- a/drivers/media/platform/qcom/iris/iris_state.h +++ b/drivers/media/platform/qcom/iris/iris_state.h @@ -91,10 +91,45 @@ enum iris_inst_state { IRIS_INST_STREAMING, IRIS_INST_ERROR, }; + +#define IRIS_INST_SUB_NONE 0 +#define IRIS_INST_SUB_STATES 8 +#define IRIS_INST_MAX_SUB_STATE_VALUE ((1 << IRIS_INST_SUB_STATES) - 1) + +/** + * enum iris_inst_sub_state + * + * IRIS_INST_SUB_FIRST_IPSC: indicates source change is received from firmware + * when output port is not yet streaming. + * IRIS_INST_SUB_DRC: indicates source change is received from firmware + * when output port is streaming and source change event is + * sent to client. + * IRIS_INST_SUB_DRC_LAST: indicates last buffer is received from firmware + * as part of source change. + * IRIS_INST_SUB_INPUT_PAUSE: source change is received form firmware. This + * indicates that firmware is paused to process + * any further input frames. + * IRIS_INST_SUB_OUTPUT_PAUSE: last buffer is received form firmware as part + * of drc sequence. This indicates that + * firmware is paused to process any further output frames. + */ +enum iris_inst_sub_state { + IRIS_INST_SUB_FIRST_IPSC = BIT(0), + IRIS_INST_SUB_DRC = BIT(1), + IRIS_INST_SUB_DRC_LAST = BIT(2), + IRIS_INST_SUB_INPUT_PAUSE = BIT(3), + IRIS_INST_SUB_OUTPUT_PAUSE = BIT(4), +}; + void iris_change_core_state(struct iris_core *core, enum iris_core_state request_state); int iris_inst_change_state(struct iris_inst *inst, enum iris_inst_state request_state); +int iris_inst_change_sub_state(struct iris_inst *inst, + enum iris_inst_sub_state clear_sub_state, + enum iris_inst_sub_state set_sub_state); int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane); int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane); +int iris_inst_sub_state_change_drc(struct iris_inst *inst); +int iris_inst_sub_state_change_drc_last(struct iris_inst *inst); #endif diff --git a/drivers/media/platform/qcom/iris/iris_vb2.c b/drivers/media/platform/qcom/iris/iris_vb2.c index 391c8225923d..cc9d5f14d125 100644 --- a/drivers/media/platform/qcom/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/iris/iris_vb2.c @@ -4,6 +4,7 @@ */ #include +#include #include #include "iris_buffer.h" @@ -204,6 +205,7 @@ void iris_vb2_stop_streaming(struct vb2_queue *q) void iris_vb2_buf_queue(struct vb2_buffer *vb2) { + static const struct v4l2_event eos = { .type = V4L2_EVENT_EOS }; struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb2); struct v4l2_m2m_ctx *m2m_ctx; struct iris_inst *inst; @@ -226,6 +228,19 @@ void iris_vb2_buf_queue(struct vb2_buffer *vb2) goto exit; } + if (V4L2_TYPE_IS_CAPTURE(vb2->vb2_queue->type)) { + if (inst->sub_state & IRIS_INST_SUB_DRC && + inst->sub_state & IRIS_INST_SUB_DRC_LAST) { + vbuf->flags |= V4L2_BUF_FLAG_LAST; + vbuf->field = V4L2_FIELD_NONE; + vb2_set_plane_payload(vb2, 0, 0); + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE); + if (inst->subscriptions & V4L2_EVENT_EOS) + v4l2_event_queue_fh(&inst->fh, &eos); + goto exit; + } + } + v4l2_m2m_buf_queue(m2m_ctx, vbuf); ret = iris_vdec_qbuf(inst, vbuf); diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index 17afb3c123c9..2ac17688da0f 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -256,6 +256,21 @@ int iris_vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_su return ret; } +void iris_vdec_src_change(struct iris_inst *inst) +{ + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + struct v4l2_event event = {0}; + struct vb2_queue *src_q; + + src_q = v4l2_m2m_get_src_vq(m2m_ctx); + if (!vb2_is_streaming(src_q)) + return; + + event.type = V4L2_EVENT_SOURCE_CHANGE; + event.u.src_change.changes = V4L2_EVENT_SRC_CH_RESOLUTION; + v4l2_event_queue_fh(&inst->fh, &event); +} + static int iris_vdec_get_num_queued_buffers(struct iris_inst *inst, enum iris_buffer_type type) { diff --git a/drivers/media/platform/qcom/iris/iris_vdec.h b/drivers/media/platform/qcom/iris/iris_vdec.h index 51bf0b96d9f0..a5d63d6f7723 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.h +++ b/drivers/media/platform/qcom/iris/iris_vdec.h @@ -14,6 +14,7 @@ int iris_vdec_enum_fmt(struct iris_inst *inst, struct v4l2_fmtdesc *f); int iris_vdec_try_fmt(struct iris_inst *inst, struct v4l2_format *f); int iris_vdec_s_fmt(struct iris_inst *inst, struct v4l2_format *f); int iris_vdec_subscribe_event(struct iris_inst *inst, const struct v4l2_event_subscription *sub); +void iris_vdec_src_change(struct iris_inst *inst); int iris_vdec_streamon_input(struct iris_inst *inst); int iris_vdec_streamon_output(struct iris_inst *inst); int iris_vdec_qbuf(struct iris_inst *inst, struct vb2_v4l2_buffer *vbuf); From patchwork Tue Aug 27 10:05:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 824184 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B22D21A0B06; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; cv=none; b=H42+H1metY/zCoIycN3ok8o18enEEK43vIzy8yr2rBKm8m9XH2w3jh4vX6q2F2USCn4bvgO9O1+d8m+vJJGyagU5cJEvwEIK+zPFBW5wXxfAkw5y8OALKRsEEYuySAB71bO1y3LHoMMQHPZWQmakx+JS6UbxNxFLuSsFfMZYJr0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753196; c=relaxed/simple; bh=5jvqDmVTHzKH/ivW3zwQiCOKr1DlSGcIDoGPt/l1xmw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=e1U/4APuS1BOl979sPi0he9cADeelL3dnC6EZ9NvszVZJmTLs5XAwPtBa21Q1UBb+zDtFk0PQJ6DJsj0BEXIQe12GLgy2OTk8V0M1jVcCYvYBuymvNDAA9D1jr193iqojnxh6GGQDBAybkyRF8WpnEcetCErNk45zGjOGL4mdwE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=gLS973BM; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="gLS973BM" Received: by smtp.kernel.org (Postfix) with ESMTPS id 824DBC4DDE8; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753196; bh=5jvqDmVTHzKH/ivW3zwQiCOKr1DlSGcIDoGPt/l1xmw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=gLS973BMxtVptGq/mKg4J4UQoS+at0p+33yfHXVpuXypFmuTRO2KwuRZu99TRBslD 4h6RQXFUss/oKHeCHh4/RQlaG8nZ2IWyehse5c+uKW42xUcUcTIY3gU7RBzsy5J29i 3ZTqoJUXAeyeRwSfET6aKIHBcyRXvjBUfzcF9lsjCyY5jRPLIdryY+qtUu/vy89zPD UoeJQAzzzWVeOR/FKhurAo0Qs7Ylb0Xfd/xVxeeHguucvmoTGUCz2d1kbZQGKnn/Eu SZB8iFp83cwFF+a0tJfsZUr+RrJWEFszBVGcw3AQwGcEgFpRDoa92AnpodJ/DemkKa 2VsR7xvW7+W1A== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75892C54734; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:49 +0530 Subject: [PATCH v3 24/29] media: iris: handle streamoff/on from client in dynamic resolution change Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-24-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=13151; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=ryuuCkDTLMVZbpdZyXsJBdzDnzwVELULwzgLQEWBJN0=; b=f/mfkuXsS29zY7+opaiYTKES7KymnS1SGBjbXaOLjMHKmvpQeSJxL9TODsOS/nIkVfM+9xJb2 K+YamZZwCedAdQ0n2OsWky5d0R1qlPGsAcJh7qAyruuAE9kJ5UC24XE X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal The decoder is stopped after it completes the dynamic resolution change sequence. Handle VIDIOC_STREAMOFF() and VIDIOC_STREAMON() on the CAPTURE queue by client to resume the decoding process. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/iris_buffer.c | 63 +++++++++++++++++++ drivers/media/platform/qcom/iris/iris_buffer.h | 1 + drivers/media/platform/qcom/iris/iris_hfi_common.h | 2 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 10 +++ .../platform/qcom/iris/iris_hfi_gen1_defines.h | 1 + .../platform/qcom/iris/iris_hfi_gen2_command.c | 39 ++++++++++++ .../platform/qcom/iris/iris_hfi_gen2_defines.h | 2 + drivers/media/platform/qcom/iris/iris_vdec.c | 71 +++++++++++++++++++++- 8 files changed, 187 insertions(+), 2 deletions(-) diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c b/drivers/media/platform/qcom/iris/iris_buffer.c index ed97e9f145a5..3a11930a8076 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/iris/iris_buffer.c @@ -410,6 +410,52 @@ int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane) return ret; } +static int iris_release_internal_buffers(struct iris_inst *inst, + enum iris_buffer_type buffer_type) +{ + const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + struct iris_buffer *buffer, *next; + struct iris_buffers *buffers; + int ret = 0; + + buffers = &inst->buffers[buffer_type]; + + list_for_each_entry_safe(buffer, next, &buffers->list, list) { + if (buffer->attr & BUF_ATTR_PENDING_RELEASE) + continue; + if (!(buffer->attr & BUF_ATTR_QUEUED)) + continue; + ret = hfi_ops->session_release_buf(inst, buffer); + if (ret) + return ret; + buffer->attr |= BUF_ATTR_PENDING_RELEASE; + } + + return ret; +} + +static int iris_release_input_internal_buffers(struct iris_inst *inst) +{ + const struct iris_platform_data *platform_data; + const u32 *internal_buf_type; + u32 internal_buffer_count; + int ret = 0; + u32 i = 0; + + platform_data = inst->core->iris_platform_data; + + internal_buf_type = platform_data->dec_ip_int_buf_tbl; + internal_buffer_count = platform_data->dec_ip_int_buf_tbl_size; + + for (i = 0; i < internal_buffer_count; i++) { + ret = iris_release_internal_buffers(inst, internal_buf_type[i]); + if (ret) + return ret; + } + + return ret; +} + int iris_alloc_and_queue_persist_bufs(struct iris_inst *inst) { struct iris_buffer *buffer, *next; @@ -440,6 +486,23 @@ int iris_alloc_and_queue_persist_bufs(struct iris_inst *inst) return ret; } +int iris_alloc_and_queue_input_int_bufs(struct iris_inst *inst) +{ + int ret; + + iris_get_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + + ret = iris_release_input_internal_buffers(inst); + if (ret) + return ret; + + ret = iris_create_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + + return iris_queue_internal_buffers(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); +} + int iris_queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_type) { struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; diff --git a/drivers/media/platform/qcom/iris/iris_buffer.h b/drivers/media/platform/qcom/iris/iris_buffer.h index bd8b74a9fbc4..b4433a629331 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.h +++ b/drivers/media/platform/qcom/iris/iris_buffer.h @@ -108,6 +108,7 @@ int iris_queue_internal_buffers(struct iris_inst *inst, u32 plane); int iris_destroy_internal_buffer(struct iris_inst *inst, struct iris_buffer *buffer); int iris_destroy_internal_buffers(struct iris_inst *inst, u32 plane); int iris_alloc_and_queue_persist_bufs(struct iris_inst *inst); +int iris_alloc_and_queue_input_int_bufs(struct iris_inst *inst); int iris_queue_buffer(struct iris_inst *inst, struct iris_buffer *buf); int iris_queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type buf_type); int iris_vb2_buffer_done(struct iris_inst *inst, struct iris_buffer *buf); diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.h b/drivers/media/platform/qcom/iris/iris_hfi_common.h index 22b0cbfcce76..ef76207aa749 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_common.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_common.h @@ -118,6 +118,8 @@ struct iris_hfi_command_ops { int (*session_start)(struct iris_inst *inst, u32 plane); int (*session_queue_buf)(struct iris_inst *inst, struct iris_buffer *buffer); int (*session_release_buf)(struct iris_inst *inst, struct iris_buffer *buffer); + int (*session_pause)(struct iris_inst *inst, u32 plane); + int (*session_resume_drc)(struct iris_inst *inst, u32 plane); int (*session_stop)(struct iris_inst *inst, u32 plane); int (*session_close)(struct iris_inst *inst); }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c index 5edee12d6a1b..d65aacb2bb13 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c @@ -205,6 +205,15 @@ static int iris_hfi_gen1_session_stop(struct iris_inst *inst, u32 plane) return ret; } +static int iris_hfi_gen1_session_continue(struct iris_inst *inst, u32 plane) +{ + struct hfi_session_pkt packet; + + iris_hfi_gen1_packet_session_cmd(inst, &packet, HFI_CMD_SESSION_CONTINUE); + + return iris_hfi_queue_cmd_write(inst->core, &packet, packet.shdr.hdr.size); +} + static int iris_hfi_gen1_queue_input_buffer(struct iris_inst *inst, struct iris_buffer *buf) { struct hfi_session_empty_buffer_compressed_pkt ip_pkt; @@ -786,6 +795,7 @@ static const struct iris_hfi_command_ops iris_hfi_gen1_command_ops = { .session_start = iris_hfi_gen1_session_start, .session_queue_buf = iris_hfi_gen1_session_queue_buffer, .session_release_buf = iris_hfi_gen1_session_unset_buffers, + .session_resume_drc = iris_hfi_gen1_session_continue, .session_stop = iris_hfi_gen1_session_stop, .session_close = iris_hfi_gen1_session_close, }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h index 41e103744bd5..48d0d8e4dfe5 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h @@ -34,6 +34,7 @@ #define HFI_CMD_SESSION_FLUSH 0x211008 #define HFI_CMD_SESSION_RELEASE_BUFFERS 0x21100b #define HFI_CMD_SESSION_RELEASE_RESOURCES 0x21100c +#define HFI_CMD_SESSION_CONTINUE 0x21100d #define HFI_ERR_SESSION_UNSUPPORTED_SETTING 0x1008 #define HFI_ERR_SESSION_UNSUPPORTED_STREAM 0x100d diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c index ab1021c3b562..42e86d6d435c 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c @@ -785,6 +785,43 @@ static int iris_hfi_gen2_session_stop(struct iris_inst *inst, u32 plane) return iris_wait_for_session_response(inst, false); } +static int iris_hfi_gen2_session_pause(struct iris_inst *inst, u32 plane) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + + iris_hfi_gen2_packet_session_command(inst, + HFI_CMD_PAUSE, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED), + iris_hfi_gen2_get_port(plane), + inst->session_id, + HFI_PAYLOAD_NONE, + NULL, + 0); + + return iris_hfi_queue_cmd_write(inst->core, inst_hfi_gen2->packet, + inst_hfi_gen2->packet->size); +} + +static int iris_hfi_gen2_session_resume_drc(struct iris_inst *inst, u32 plane) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + u32 payload = HFI_CMD_SETTINGS_CHANGE; + + iris_hfi_gen2_packet_session_command(inst, + HFI_CMD_RESUME, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED), + iris_hfi_gen2_get_port(plane), + inst->session_id, + HFI_PAYLOAD_U32, + &payload, + sizeof(u32)); + + return iris_hfi_queue_cmd_write(inst->core, inst_hfi_gen2->packet, + inst_hfi_gen2->packet->size); +} + static u32 iris_hfi_gen2_buf_type_from_driver(enum iris_buffer_type buffer_type) { switch (buffer_type) { @@ -906,6 +943,8 @@ static const struct iris_hfi_command_ops iris_hfi_gen2_command_ops = { .session_start = iris_hfi_gen2_session_start, .session_queue_buf = iris_hfi_gen2_session_queue_buffer, .session_release_buf = iris_hfi_gen2_session_release_buffer, + .session_pause = iris_hfi_gen2_session_pause, + .session_resume_drc = iris_hfi_gen2_session_resume_drc, .session_stop = iris_hfi_gen2_session_stop, .session_close = iris_hfi_gen2_session_close, }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h index 66ffce02ae57..0aaba5f1ce91 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h @@ -17,9 +17,11 @@ #define HFI_CMD_CLOSE 0x01000004 #define HFI_CMD_START 0x01000005 #define HFI_CMD_STOP 0x01000006 +#define HFI_CMD_RESUME 0x01000008 #define HFI_CMD_BUFFER 0x01000009 #define HFI_CMD_SUBSCRIBE_MODE 0x0100000B #define HFI_CMD_SETTINGS_CHANGE 0x0100000C +#define HFI_CMD_PAUSE 0x01000011 #define HFI_CMD_END 0x01FFFFFF #define HFI_BITMASK_BITSTREAM_WIDTH 0xffff0000 diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index 2ac17688da0f..6d3b06150cf7 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -385,13 +385,39 @@ int iris_vdec_session_streamoff(struct iris_inst *inst, u32 plane) static int iris_vdec_process_streamon_input(struct iris_inst *inst) { const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE; int ret; ret = hfi_ops->session_start(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); if (ret) return ret; - return iris_inst_state_change_streamon(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = iris_inst_change_sub_state(inst, IRIS_INST_SUB_INPUT_PAUSE, 0); + if (ret) + return ret; + } + + if (inst->sub_state & IRIS_INST_SUB_DRC || + inst->sub_state & IRIS_INST_SUB_FIRST_IPSC) { + if (!(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE)) { + if (hfi_ops->session_pause) { + ret = hfi_ops->session_pause(inst, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + } + set_sub_state = IRIS_INST_SUB_INPUT_PAUSE; + } + } + + ret = iris_inst_state_change_streamon(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + + ret = iris_inst_change_sub_state(inst, 0, set_sub_state); + + return ret; } int iris_vdec_streamon_input(struct iris_inst *inst) @@ -430,13 +456,54 @@ int iris_vdec_streamon_input(struct iris_inst *inst) static int iris_vdec_process_streamon_output(struct iris_inst *inst) { const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE; + bool drc_active = false; int ret; + drc_active = inst->sub_state & IRIS_INST_SUB_DRC && + inst->sub_state & IRIS_INST_SUB_DRC_LAST; + + if (drc_active) + clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST; + + if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = iris_alloc_and_queue_input_int_bufs(inst); + if (ret) + return ret; + ret = iris_set_stage(inst, STAGE); + if (ret) + return ret; + ret = iris_set_pipe(inst, PIPE); + if (ret) + return ret; + } + + if (inst->state == IRIS_INST_INPUT_STREAMING && + inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = hfi_ops->session_resume_drc(inst, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE; + } + + if (inst->sub_state & IRIS_INST_SUB_FIRST_IPSC) + clear_sub_state |= IRIS_INST_SUB_FIRST_IPSC; + ret = hfi_ops->session_start(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); if (ret) return ret; - return iris_inst_state_change_streamon(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) + clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE; + + ret = iris_inst_state_change_streamon(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + if (ret) + return ret; + + ret = iris_inst_change_sub_state(inst, clear_sub_state, 0); + + return ret; } int iris_vdec_streamon_output(struct iris_inst *inst) From patchwork Tue Aug 27 10:05:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 822718 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 05D2E1A256A; Tue, 27 Aug 2024 10:06:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753197; cv=none; b=tQZNS1GJru4u35z6+i/nlGcaPdBw7ZuB6cOUgEn6+f5pGjqMD3MyD+yvbJEZBz/qVZnRn0wIu+9z+qvCEICwDa+tKBx+9XuuVHNo8SRrCVMRVTh0aGKbyeinf47ao1aVkjGLT9VobpD/WxkKyg/hW/G/wSOKAAoKcTtdU4oGTeI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753197; c=relaxed/simple; bh=lEsZ8VdAZIMGM4/Gx/+jsJ4lARBe5/lCLdrttajtXVI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=aR+HmCKHVdEspXXoCkybWh07amkH4jsMv9Qi59HBck1xMail0BrSRV/eblLT7Yr6rAoANeZ9KWvEu4dQTehdTT9+a5g4Sl82nT/0KS+PoAIGoXUa7XLiK0B52lTH2GBuypw1TmZRCxF15NHpDaIWt9GqHPdjfZzT4rkW+qBpowk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=juMi7WWy; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="juMi7WWy" Received: by smtp.kernel.org (Postfix) with ESMTPS id 94E11C4FEB0; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753196; bh=lEsZ8VdAZIMGM4/Gx/+jsJ4lARBe5/lCLdrttajtXVI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=juMi7WWyx6RdqTusr0vIO+Z4WZipP6ZkH+OSbGc+VeEITzEzZobQ8hYBH37c9B9hi GTLfhVIHRpqJcjNNBBFnSD+hUDSxV4tKtfnnNPulliqTD0FJX5K8B3RpYzbelcxqD5 tvY2asyKO2OZDaYDRg76j5DhB0qGo3a+6869gqhJgTQ/0WUbkye6nxIqqvyffqChb+ 49aXmQ29YW/GfxsWypn+/Hbd4eYfaVHUI0ifAhLzMKUiPnOIkqih1OSU0grrZyVq3a onOKm82hl+Ho6AWF7bcBCCSRKLe0s8l0+tdireEdGsasElCn8XtfT30nQw+Etqrm25 CaYokdZOtUIFw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89028C54736; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:50 +0530 Subject: [PATCH v3 25/29] media: iris: add support for drain sequence Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-25-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=23616; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=o/SpScyY2OTjOGR+Ow4+9nzhdiAuBrhMDfgaB2nN4lg=; b=TXR5abSlYWO7CYYPSj/d4thqd8m1ZsxTnom/IuO3wYup3JVgpyqY5xfJKOCcQbQ6Udo2oK3Co SdKwgEJLACnBfKFBcnXXGd8fY6EmQfGHL2WeBm5WixMQ/3C/ZKLjheE X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Dikshita Agarwal handle the V4L2_DEC_CMD_STOP by initiating drain sequence to firmware. Process and decode all OUTPUT buffers queued by the client before the VIDIOC_DECODER_CMD() was issued and mark the last buffer with V4L2_BUF_FLAG_LAST flag. Decoder is stopped after processing the last buffer. Resume the decoder when one of below are issued by client: - V4L2_DEC_CMD_START - pair of VIDIOC_STREAMOFF() and VIDIOC_STREAMON() on the CAPTURE queue - pair of VIDIOC_STREAMOFF() and VIDIOC_STREAMON() on the OUTPUT queue Add the handling to resume decoding when client issues V4L2_DEC_CMD_START to resume decoding after source change is detected. Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/iris_hfi_common.h | 2 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 13 +++ .../platform/qcom/iris/iris_hfi_gen1_defines.h | 1 + .../platform/qcom/iris/iris_hfi_gen1_response.c | 14 +++ .../platform/qcom/iris/iris_hfi_gen2_command.c | 43 +++++++++ .../platform/qcom/iris/iris_hfi_gen2_defines.h | 2 + .../platform/qcom/iris/iris_hfi_gen2_response.c | 46 ++++++++- drivers/media/platform/qcom/iris/iris_state.c | 36 +++++++ drivers/media/platform/qcom/iris/iris_state.h | 11 ++- drivers/media/platform/qcom/iris/iris_vb2.c | 6 +- drivers/media/platform/qcom/iris/iris_vdec.c | 105 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_vdec.h | 2 + drivers/media/platform/qcom/iris/iris_vidc.c | 36 +++++++ 13 files changed, 307 insertions(+), 10 deletions(-) diff --git a/drivers/media/platform/qcom/iris/iris_hfi_common.h b/drivers/media/platform/qcom/iris/iris_hfi_common.h index ef76207aa749..5bd7073a2d9b 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_common.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_common.h @@ -121,6 +121,8 @@ struct iris_hfi_command_ops { int (*session_pause)(struct iris_inst *inst, u32 plane); int (*session_resume_drc)(struct iris_inst *inst, u32 plane); int (*session_stop)(struct iris_inst *inst, u32 plane); + int (*session_drain)(struct iris_inst *inst, u32 plane); + int (*session_resume_drain)(struct iris_inst *inst, u32 plane); int (*session_close)(struct iris_inst *inst); }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c index d65aacb2bb13..17333768afa2 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c @@ -369,6 +369,18 @@ static int iris_hfi_gen1_session_unset_buffers(struct iris_inst *inst, struct ir return ret; } +static int iris_hfi_gen1_session_drain(struct iris_inst *inst, u32 plane) +{ + struct hfi_session_empty_buffer_compressed_pkt ip_pkt = {0}; + + ip_pkt.shdr.hdr.size = sizeof(struct hfi_session_empty_buffer_compressed_pkt); + ip_pkt.shdr.hdr.pkt_type = HFI_CMD_SESSION_EMPTY_BUFFER; + ip_pkt.shdr.session_id = inst->session_id; + ip_pkt.flags = HFI_BUFFERFLAG_EOS; + + return iris_hfi_queue_cmd_write(inst->core, &ip_pkt, ip_pkt.shdr.hdr.size); +} + static int iris_hfi_gen1_packet_session_set_property(struct hfi_session_set_property_pkt *packet, struct iris_inst *inst, u32 ptype, void *pdata) @@ -797,6 +809,7 @@ static const struct iris_hfi_command_ops iris_hfi_gen1_command_ops = { .session_release_buf = iris_hfi_gen1_session_unset_buffers, .session_resume_drc = iris_hfi_gen1_session_continue, .session_stop = iris_hfi_gen1_session_stop, + .session_drain = iris_hfi_gen1_session_drain, .session_close = iris_hfi_gen1_session_close, }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h index 48d0d8e4dfe5..7f910fa0bfd7 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h @@ -49,6 +49,7 @@ #define HFI_EVENT_DATA_SEQUENCE_CHANGED_INSUFFICIENT_BUF_RESOURCES 0x1000002 #define HFI_EVENT_SESSION_SEQUENCE_CHANGED 0x1000003 +#define HFI_BUFFERFLAG_EOS 0x00000001 #define HFI_BUFFERFLAG_TIMESTAMPINVALID 0x00000100 #define HFI_FLUSH_OUTPUT 0x1000002 diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c index e3d9bfb8bef6..2d537babd3e7 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c @@ -378,6 +378,7 @@ static void iris_hfi_gen1_session_ftb_done(struct iris_inst *inst, void *packet) struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; struct v4l2_m2m_buffer *m2m_buffer, *n; u32 timestamp_hi = 0, timestamp_lo = 0; + struct hfi_session_flush_pkt flush_pkt; struct iris_buffer *buf = NULL, *iter; struct iris_core *core = inst->core; u32 pic_type = 0, output_tag = -1; @@ -393,6 +394,19 @@ static void iris_hfi_gen1_session_ftb_done(struct iris_inst *inst, void *packet) pic_type = pkt->picture_type; output_tag = pkt->output_tag; + if ((hfi_flags & HFI_BUFFERFLAG_EOS) && !filled_len) { + reinit_completion(&inst->flush_completion); + + flush_pkt.shdr.hdr.size = sizeof(struct hfi_session_flush_pkt); + flush_pkt.shdr.hdr.pkt_type = HFI_CMD_SESSION_FLUSH; + flush_pkt.shdr.session_id = inst->session_id; + flush_pkt.flush_type = HFI_FLUSH_OUTPUT; + iris_hfi_queue_cmd_write(core, &flush_pkt, flush_pkt.shdr.hdr.size); + iris_inst_sub_state_change_drain_last(inst); + + return; + } + if (iris_split_mode_enabled(inst) && pkt->stream_id == 0) { buffers = &inst->buffers[BUF_DPB]; if (!buffers) diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c index 42e86d6d435c..ee73a2b072db 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c @@ -822,6 +822,47 @@ static int iris_hfi_gen2_session_resume_drc(struct iris_inst *inst, u32 plane) inst_hfi_gen2->packet->size); } +static int iris_hfi_gen2_session_resume_drain(struct iris_inst *inst, u32 plane) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + u32 payload = HFI_CMD_DRAIN; + + iris_hfi_gen2_packet_session_command(inst, + HFI_CMD_RESUME, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED), + iris_hfi_gen2_get_port(plane), + inst->session_id, + HFI_PAYLOAD_U32, + &payload, + sizeof(u32)); + + return iris_hfi_queue_cmd_write(inst->core, inst_hfi_gen2->packet, + inst_hfi_gen2->packet->size); +} + +static int iris_hfi_gen2_session_drain(struct iris_inst *inst, u32 plane) +{ + struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); + + if (!V4L2_TYPE_IS_OUTPUT(plane)) + return 0; + + iris_hfi_gen2_packet_session_command(inst, + HFI_CMD_DRAIN, + (HFI_HOST_FLAGS_RESPONSE_REQUIRED | + HFI_HOST_FLAGS_INTR_REQUIRED | + HFI_HOST_FLAGS_NON_DISCARDABLE), + iris_hfi_gen2_get_port(plane), + inst->session_id, + HFI_PAYLOAD_NONE, + NULL, + 0); + + return iris_hfi_queue_cmd_write(inst->core, inst_hfi_gen2->packet, + inst_hfi_gen2->packet->size); +} + static u32 iris_hfi_gen2_buf_type_from_driver(enum iris_buffer_type buffer_type) { switch (buffer_type) { @@ -946,6 +987,8 @@ static const struct iris_hfi_command_ops iris_hfi_gen2_command_ops = { .session_pause = iris_hfi_gen2_session_pause, .session_resume_drc = iris_hfi_gen2_session_resume_drc, .session_stop = iris_hfi_gen2_session_stop, + .session_drain = iris_hfi_gen2_session_drain, + .session_resume_drain = iris_hfi_gen2_session_resume_drain, .session_close = iris_hfi_gen2_session_close, }; diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h index 0aaba5f1ce91..72d41445d765 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h @@ -17,6 +17,7 @@ #define HFI_CMD_CLOSE 0x01000004 #define HFI_CMD_START 0x01000005 #define HFI_CMD_STOP 0x01000006 +#define HFI_CMD_DRAIN 0x01000007 #define HFI_CMD_RESUME 0x01000008 #define HFI_CMD_BUFFER 0x01000009 #define HFI_CMD_SUBSCRIBE_MODE 0x0100000B @@ -79,6 +80,7 @@ #define HFI_INFO_UNSUPPORTED 0x06000001 #define HFI_INFO_DATA_CORRUPT 0x06000002 #define HFI_INFO_BUFFER_OVERFLOW 0x06000004 +#define HFI_INFO_HFI_FLAG_DRAIN_LAST 0x06000006 #define HFI_INFO_HFI_FLAG_PSC_LAST 0x06000007 #define HFI_INFORMATION_END 0x06FFFFFF diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c index 626a858e803c..8b3402635674 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c @@ -202,6 +202,10 @@ static int iris_hfi_gen2_handle_session_info(struct iris_inst *inst, info = "buffer overflow"; inst_hfi_gen2->hfi_frame_info.overflow = 1; break; + case HFI_INFO_HFI_FLAG_DRAIN_LAST: + info = "drain last flag"; + ret = iris_inst_sub_state_change_drain_last(inst); + break; case HFI_INFO_HFI_FLAG_PSC_LAST: info = "drc last flag"; ret = iris_inst_sub_state_change_drc_last(inst); @@ -329,6 +333,12 @@ static int iris_hfi_gen2_handle_output_buffer(struct iris_inst *inst, int ret = 0; bool found = false; + if (hfi_buffer->flags & HFI_BUF_FW_FLAG_LAST) { + ret = iris_inst_sub_state_change_drain_last(inst); + if (ret) + return ret; + } + if (hfi_buffer->flags & HFI_BUF_FW_FLAG_PSC_LAST) { ret = iris_inst_sub_state_change_drc_last(inst); if (ret) @@ -429,6 +439,21 @@ static int iris_hfi_gen2_handle_release_internal_buffer(struct iris_inst *inst, return ret; } +static int iris_hfi_gen2_handle_session_stop(struct iris_inst *inst, + struct iris_hfi_packet *pkt) +{ + int ret = 0; + + if (pkt->port == HFI_PORT_RAW) + ret = iris_inst_sub_state_change_pause(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + else if (pkt->port == HFI_PORT_BITSTREAM) + ret = iris_inst_sub_state_change_pause(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + + complete(&inst->completion); + + return ret; +} + static int iris_hfi_gen2_handle_session_buffer(struct iris_inst *inst, struct iris_hfi_packet *pkt) { @@ -457,6 +482,22 @@ static int iris_hfi_gen2_handle_session_buffer(struct iris_inst *inst, return iris_hfi_gen2_handle_release_internal_buffer(inst, buffer); } +static int iris_hfi_gen2_handle_session_drain(struct iris_inst *inst, + struct iris_hfi_packet *pkt) +{ + int ret = 0; + + if (!(pkt->flags & HFI_FW_FLAGS_SUCCESS)) { + iris_inst_change_state(inst, IRIS_INST_ERROR); + return 0; + } + + if (inst->sub_state & IRIS_INST_SUB_DRAIN) + ret = iris_inst_change_sub_state(inst, 0, IRIS_INST_SUB_INPUT_PAUSE); + + return ret; +} + static void iris_hfi_gen2_read_input_subcr_params(struct iris_inst *inst) { struct iris_inst_hfi_gen2 *inst_hfi_gen2 = to_iris_inst_hfi_gen2(inst); @@ -569,7 +610,7 @@ static int iris_hfi_gen2_handle_session_command(struct iris_inst *inst, iris_hfi_gen2_handle_session_close(inst, pkt); break; case HFI_CMD_STOP: - complete(&inst->completion); + iris_hfi_gen2_handle_session_stop(inst, pkt); break; case HFI_CMD_BUFFER: ret = iris_hfi_gen2_handle_session_buffer(inst, pkt); @@ -577,6 +618,9 @@ static int iris_hfi_gen2_handle_session_command(struct iris_inst *inst, case HFI_CMD_SETTINGS_CHANGE: ret = iris_hfi_gen2_handle_src_change(inst, pkt); break; + case HFI_CMD_DRAIN: + ret = iris_hfi_gen2_handle_session_drain(inst, pkt); + break; default: break; } diff --git a/drivers/media/platform/qcom/iris/iris_state.c b/drivers/media/platform/qcom/iris/iris_state.c index e39c110b75e0..d37f99d72491 100644 --- a/drivers/media/platform/qcom/iris/iris_state.c +++ b/drivers/media/platform/qcom/iris/iris_state.c @@ -160,6 +160,21 @@ int iris_inst_sub_state_change_drc(struct iris_inst *inst) return iris_inst_change_sub_state(inst, 0, set_sub_state); } +int iris_inst_sub_state_change_drain_last(struct iris_inst *inst) +{ + enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE; + + if (inst->sub_state & IRIS_INST_SUB_DRAIN_LAST) + return -EINVAL; + + if (!(inst->sub_state & IRIS_INST_SUB_DRAIN)) + return -EINVAL; + + set_sub_state = IRIS_INST_SUB_DRAIN_LAST | IRIS_INST_SUB_OUTPUT_PAUSE; + + return iris_inst_change_sub_state(inst, 0, set_sub_state); +} + int iris_inst_sub_state_change_drc_last(struct iris_inst *inst) { enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE; @@ -178,3 +193,24 @@ int iris_inst_sub_state_change_drc_last(struct iris_inst *inst) return iris_inst_change_sub_state(inst, 0, set_sub_state); } + +int iris_inst_sub_state_change_pause(struct iris_inst *inst, u32 plane) +{ + enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE; + + if (V4L2_TYPE_IS_OUTPUT(plane)) { + if (inst->sub_state & IRIS_INST_SUB_DRC && + !(inst->sub_state & IRIS_INST_SUB_DRC_LAST)) + return -EINVAL; + + if (inst->sub_state & IRIS_INST_SUB_DRAIN && + !(inst->sub_state & IRIS_INST_SUB_DRAIN_LAST)) + return -EINVAL; + + set_sub_state = IRIS_INST_SUB_INPUT_PAUSE; + } else { + set_sub_state = IRIS_INST_SUB_OUTPUT_PAUSE; + } + + return iris_inst_change_sub_state(inst, 0, set_sub_state); +} diff --git a/drivers/media/platform/qcom/iris/iris_state.h b/drivers/media/platform/qcom/iris/iris_state.h index 786b82ca0307..a01037918a9c 100644 --- a/drivers/media/platform/qcom/iris/iris_state.h +++ b/drivers/media/platform/qcom/iris/iris_state.h @@ -106,6 +106,9 @@ enum iris_inst_state { * sent to client. * IRIS_INST_SUB_DRC_LAST: indicates last buffer is received from firmware * as part of source change. + * IRIS_INST_SUB_DRAIN: indicates drain is in progress. + * IRIS_INST_SUB_DRAIN_LAST: indicates last buffer is received from firmware + * as part of drain sequence. * IRIS_INST_SUB_INPUT_PAUSE: source change is received form firmware. This * indicates that firmware is paused to process * any further input frames. @@ -117,8 +120,10 @@ enum iris_inst_sub_state { IRIS_INST_SUB_FIRST_IPSC = BIT(0), IRIS_INST_SUB_DRC = BIT(1), IRIS_INST_SUB_DRC_LAST = BIT(2), - IRIS_INST_SUB_INPUT_PAUSE = BIT(3), - IRIS_INST_SUB_OUTPUT_PAUSE = BIT(4), + IRIS_INST_SUB_DRAIN = BIT(3), + IRIS_INST_SUB_DRAIN_LAST = BIT(4), + IRIS_INST_SUB_INPUT_PAUSE = BIT(5), + IRIS_INST_SUB_OUTPUT_PAUSE = BIT(6), }; void iris_change_core_state(struct iris_core *core, @@ -131,5 +136,7 @@ int iris_inst_change_sub_state(struct iris_inst *inst, int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane); int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane); int iris_inst_sub_state_change_drc(struct iris_inst *inst); +int iris_inst_sub_state_change_drain_last(struct iris_inst *inst); int iris_inst_sub_state_change_drc_last(struct iris_inst *inst); +int iris_inst_sub_state_change_pause(struct iris_inst *inst, u32 plane); #endif diff --git a/drivers/media/platform/qcom/iris/iris_vb2.c b/drivers/media/platform/qcom/iris/iris_vb2.c index cc9d5f14d125..8bed24cf610e 100644 --- a/drivers/media/platform/qcom/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/iris/iris_vb2.c @@ -229,8 +229,10 @@ void iris_vb2_buf_queue(struct vb2_buffer *vb2) } if (V4L2_TYPE_IS_CAPTURE(vb2->vb2_queue->type)) { - if (inst->sub_state & IRIS_INST_SUB_DRC && - inst->sub_state & IRIS_INST_SUB_DRC_LAST) { + if ((inst->sub_state & IRIS_INST_SUB_DRC && + inst->sub_state & IRIS_INST_SUB_DRC_LAST) || + (inst->sub_state & IRIS_INST_SUB_DRAIN && + inst->sub_state & IRIS_INST_SUB_DRAIN_LAST)) { vbuf->flags |= V4L2_BUF_FLAG_LAST; vbuf->field = V4L2_FIELD_NONE; vb2_set_plane_payload(vb2, 0, 0); diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index 6d3b06150cf7..b8d2d6846daa 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -399,6 +399,7 @@ static int iris_vdec_process_streamon_input(struct iris_inst *inst) } if (inst->sub_state & IRIS_INST_SUB_DRC || + inst->sub_state & IRIS_INST_SUB_DRAIN || inst->sub_state & IRIS_INST_SUB_FIRST_IPSC) { if (!(inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE)) { if (hfi_ops->session_pause) { @@ -457,14 +458,19 @@ static int iris_vdec_process_streamon_output(struct iris_inst *inst) { const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE; - bool drc_active = false; + bool drain_active = false, drc_active = false; int ret; + drain_active = inst->sub_state & IRIS_INST_SUB_DRAIN && + inst->sub_state & IRIS_INST_SUB_DRAIN_LAST; + drc_active = inst->sub_state & IRIS_INST_SUB_DRC && inst->sub_state & IRIS_INST_SUB_DRC_LAST; if (drc_active) clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST; + else if (drain_active) + clear_sub_state = IRIS_INST_SUB_DRAIN | IRIS_INST_SUB_DRAIN_LAST; if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { ret = iris_alloc_and_queue_input_int_bufs(inst); @@ -480,10 +486,20 @@ static int iris_vdec_process_streamon_output(struct iris_inst *inst) if (inst->state == IRIS_INST_INPUT_STREAMING && inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { - ret = hfi_ops->session_resume_drc(inst, - V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); - if (ret) - return ret; + if (!drain_active) { + ret = hfi_ops->session_resume_drc(inst, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + } else { + if (hfi_ops->session_resume_drain) { + ret = + hfi_ops->session_resume_drain(inst, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + } + } clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE; } @@ -584,3 +600,82 @@ int iris_vdec_qbuf(struct iris_inst *inst, struct vb2_v4l2_buffer *vbuf) return iris_queue_buffer(inst, buf); } + +int iris_vdec_start_cmd(struct iris_inst *inst) +{ + const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + enum iris_inst_sub_state clear_sub_state = IRIS_INST_SUB_NONE; + struct vb2_queue *dst_vq; + int ret; + + dst_vq = v4l2_m2m_get_vq(inst->m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + + if (inst->sub_state & IRIS_INST_SUB_DRC && + inst->sub_state & IRIS_INST_SUB_DRC_LAST) { + vb2_clear_last_buffer_dequeued(dst_vq); + clear_sub_state = IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRC_LAST; + + if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + ret = hfi_ops->session_resume_drc(inst, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE; + } + if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) { + ret = hfi_ops->session_resume_drc(inst, + V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + if (ret) + return ret; + clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE; + } + } else if (inst->sub_state & IRIS_INST_SUB_DRAIN && + inst->sub_state & IRIS_INST_SUB_DRAIN_LAST) { + vb2_clear_last_buffer_dequeued(dst_vq); + clear_sub_state = IRIS_INST_SUB_DRAIN | IRIS_INST_SUB_DRAIN_LAST; + if (inst->sub_state & IRIS_INST_SUB_INPUT_PAUSE) { + if (hfi_ops->session_resume_drain) { + ret = + hfi_ops->session_resume_drain(inst, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + } + + clear_sub_state |= IRIS_INST_SUB_INPUT_PAUSE; + } + if (inst->sub_state & IRIS_INST_SUB_OUTPUT_PAUSE) { + if (hfi_ops->session_resume_drain) { + ret = + hfi_ops->session_resume_drain(inst, + V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + if (ret) + return ret; + } + + clear_sub_state |= IRIS_INST_SUB_OUTPUT_PAUSE; + } + } else { + dev_err(inst->core->dev, "start called before receiving last_flag\n"); + iris_inst_change_state(inst, IRIS_INST_ERROR); + return -EBUSY; + } + + ret = iris_inst_change_sub_state(inst, clear_sub_state, 0); + + return ret; +} + +int iris_vdec_stop_cmd(struct iris_inst *inst) +{ + const struct iris_hfi_command_ops *hfi_ops = inst->core->hfi_ops; + int ret; + + ret = hfi_ops->session_drain(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (ret) + return ret; + + iris_inst_change_sub_state(inst, 0, IRIS_INST_SUB_DRAIN); + + return ret; +} diff --git a/drivers/media/platform/qcom/iris/iris_vdec.h b/drivers/media/platform/qcom/iris/iris_vdec.h index a5d63d6f7723..1d0231db24de 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.h +++ b/drivers/media/platform/qcom/iris/iris_vdec.h @@ -18,6 +18,8 @@ void iris_vdec_src_change(struct iris_inst *inst); int iris_vdec_streamon_input(struct iris_inst *inst); int iris_vdec_streamon_output(struct iris_inst *inst); int iris_vdec_qbuf(struct iris_inst *inst, struct vb2_v4l2_buffer *vbuf); +int iris_vdec_start_cmd(struct iris_inst *inst); +int iris_vdec_stop_cmd(struct iris_inst *inst); int iris_vdec_session_streamoff(struct iris_inst *inst, u32 plane); #endif diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index 34c2548bce9f..07eb276472f6 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -603,6 +603,40 @@ static int iris_unsubscribe_event(struct v4l2_fh *fh, const struct v4l2_event_su return ret; } +static int iris_dec_cmd(struct file *filp, void *fh, + struct v4l2_decoder_cmd *dec) +{ + struct iris_inst *inst; + int ret = 0; + + inst = iris_get_inst(filp, fh); + if (!inst || !dec) + return -EINVAL; + + mutex_lock(&inst->lock); + + if (dec->cmd != V4L2_DEC_CMD_START && + dec->cmd != V4L2_DEC_CMD_STOP) { + ret = -EINVAL; + goto unlock; + } + + if (inst->state == IRIS_INST_DEINIT) + goto unlock; + + if (dec->cmd == V4L2_DEC_CMD_START) + ret = iris_vdec_start_cmd(inst); + else if (dec->cmd == V4L2_DEC_CMD_STOP) + ret = iris_vdec_stop_cmd(inst); + else + ret = -EINVAL; + +unlock: + mutex_unlock(&inst->lock); + + return ret; +} + static struct v4l2_file_operations iris_v4l2_file_ops = { .owner = THIS_MODULE, .open = iris_open, @@ -645,6 +679,8 @@ static const struct v4l2_ioctl_ops iris_v4l2_ioctl_ops = { .vidioc_unsubscribe_event = iris_unsubscribe_event, .vidioc_streamon = v4l2_m2m_ioctl_streamon, .vidioc_streamoff = v4l2_m2m_ioctl_streamoff, + .vidioc_try_decoder_cmd = v4l2_m2m_ioctl_try_decoder_cmd, + .vidioc_decoder_cmd = iris_dec_cmd, }; void iris_init_ops(struct iris_core *core) From patchwork Tue Aug 27 10:05:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 824183 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1AF3E1A2844; Tue, 27 Aug 2024 10:06:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753197; cv=none; b=uKXnI7JAJdl1JYm8pD7EXOL69VopBsQDAsS7CjeMvdclBGQz4mAWjDAF1+HR67Mt/SigzPvFkAuv9UwmiNdjr4xahLh4W24gxcV5+/vj4pezkqCu9xv//0vtQCHZ3mNY70Iy+k+Y02RYS7faEV4UOmE4CGaLvm1qxoneAnXoJRk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753197; c=relaxed/simple; bh=Soq+mY6Aeg5XklMm/TtlVPGJv2lXW870lFp6r7aI1bM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=sXkICqodR5jqXcaXo46A3TfnDp8NYj+d7Oi7QUncwd7XoFUdkQkFmxOq+WZ3UPvMF1ZgcKAs2sgqdGlY716lMt8swQiEiX24ydML5/SW2XNBQC7LXNiXc8d+O8/EMDSlLlaQ5p7rQnoynBo0RqU98kVC2kmROQsV9KhVAeSt9Y4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=s2uxc6uL; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="s2uxc6uL" Received: by smtp.kernel.org (Postfix) with ESMTPS id C4AFEC4DDE7; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753196; bh=Soq+mY6Aeg5XklMm/TtlVPGJv2lXW870lFp6r7aI1bM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=s2uxc6uLoJZy6tX3EU94FvvOxG4ZdTnOtHtDrAkWMZlRITuTjvj2Ain6/i5pkv18U 6WQ4X7QXjtaK7MrMSIcCETfLcClymbfBP0kAR6KMx1yCBwsF6SYbwJHv91mTqgVc0G 6baBoXllbr3HmsUYbq1fn/2sVdBdaTXVCpMO+Ae5Q5MCzB4CL/TYRJbQTCyMe+OoJd nZgmtACKH/M9tReJ5uKc7967FjjMG1mt+WnskRi2AiWEA9l5KxPR2O/i5E/Y0T7XIp yqVJWIHaJwUhluLkDmCtVJ6r/9tLXOq+nQmNBjKw+ezGhen/FVuhzAe9a6zI7E84Nb VdX7zSINxJKTg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id B87DCC54735; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:51 +0530 Subject: [PATCH v3 26/29] media: iris: add check whether the video session is supported or not Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-26-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal , Vedang Nagar X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=5019; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=j4ih5G3BXzUsyLq/oyA3xJ7gNC6riTH012ozApP83VA=; b=rJSmQ2TeLNiwhyNXJuJ59yVGLGOBqnofra9hEXjdBSjPtL5j9DNCVECKcKq75tBZN5VV90zCR QiMkx/I4560DJC1Kjog7e/FczsSSeyFdWZPOw/kaT0GkiWwtCxuPj32 X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Vedang Nagar Based on the hardware capabilities, add check to restrict whether the video session is supported or not by the hardware during start_streaming and queue_setup. Signed-off-by: Vedang Nagar Signed-off-by: Dikshita Agarwal --- .../platform/qcom/iris/iris_platform_common.h | 1 + .../platform/qcom/iris/iris_platform_sm8250.c | 1 + .../platform/qcom/iris/iris_platform_sm8550.c | 1 + drivers/media/platform/qcom/iris/iris_vb2.c | 80 ++++++++++++++++++++++ 4 files changed, 83 insertions(+) diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h index 30bc3c0e57a2..2abc96d56a4b 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_common.h +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h @@ -156,6 +156,7 @@ struct iris_platform_data { struct ubwc_config_data *ubwc_config; u32 num_vpp_pipe; u32 max_session_count; + u32 max_mbpf; const u32 *input_config_params; unsigned int input_config_params_size; const u32 *output_config_params; diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c index 8316f7081ce0..81f8366bcdd2 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c @@ -141,6 +141,7 @@ struct iris_platform_data sm8250_data = { .hw_response_timeout = HW_RESPONSE_TIMEOUT_VALUE, .num_vpp_pipe = 4, .max_session_count = 16, + .max_mbpf = (8192 * 4352) / 256, .input_config_params = sm8250_vdec_input_config_param, .input_config_params_size = diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c index 3d84e1a6796c..1b7644fdd21b 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c @@ -262,6 +262,7 @@ struct iris_platform_data sm8550_data = { .ubwc_config = &ubwc_config_sm8550, .num_vpp_pipe = 4, .max_session_count = 16, + .max_mbpf = ((8192 * 4352) / 256) * 2, .input_config_params = sm8550_vdec_input_config_params, .input_config_params_size = diff --git a/drivers/media/platform/qcom/iris/iris_vb2.c b/drivers/media/platform/qcom/iris/iris_vb2.c index 8bed24cf610e..9dfbd0d66b61 100644 --- a/drivers/media/platform/qcom/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/iris/iris_vb2.c @@ -12,6 +12,78 @@ #include "iris_vb2.h" #include "iris_vdec.h" #include "iris_vpu_buffer.h" + +static int iris_check_core_mbpf(struct iris_inst *inst) +{ + struct iris_core *core = inst->core; + struct iris_inst *instance; + u32 total_mbpf = 0; + + mutex_lock(&core->lock); + list_for_each_entry(instance, &core->instances, list) + total_mbpf += iris_get_mbpf(instance); + mutex_unlock(&core->lock); + + if (total_mbpf > core->iris_platform_data->max_mbpf) + return -ENOMEM; + + return 0; +} + +static int iris_check_inst_mbpf(struct iris_inst *inst) +{ + u32 mbpf, max_mbpf; + + max_mbpf = inst->driver_cap[MBPF].max; + mbpf = iris_get_mbpf(inst); + if (mbpf > max_mbpf) + return -ENOMEM; + + return 0; +} + +static int iris_check_resolution_supported(struct iris_inst *inst) +{ + u32 width, height, min_width, min_height, max_width, max_height; + + width = inst->fmt_src->fmt.pix_mp.width; + height = inst->fmt_src->fmt.pix_mp.height; + + min_width = inst->driver_cap[FRAME_WIDTH].min; + max_width = inst->driver_cap[FRAME_WIDTH].max; + min_height = inst->driver_cap[FRAME_HEIGHT].min; + max_height = inst->driver_cap[FRAME_HEIGHT].max; + + if (!(min_width <= width && width <= max_width) || + !(min_height <= height && height <= max_height)) + return -EINVAL; + + return 0; +} + +static int iris_check_session_supported(struct iris_inst *inst) +{ + int ret; + + ret = iris_check_core_mbpf(inst); + if (ret) + goto exit; + + ret = iris_check_inst_mbpf(inst); + if (ret) + goto exit; + + ret = iris_check_resolution_supported(inst); + if (ret) + goto exit; + + return 0; +exit: + dev_err(inst->core->dev, "current session not supported(%d)\n", ret); + + return ret; +} + int iris_vb2_buf_init(struct vb2_buffer *vb2) { struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb2); @@ -69,6 +141,10 @@ int iris_vb2_queue_setup(struct vb2_queue *q, goto unlock; } + ret = iris_check_session_supported(inst); + if (ret) + goto unlock; + if (!inst->once_per_session_set) { inst->once_per_session_set = true; @@ -144,6 +220,10 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count) goto error; } + ret = iris_check_session_supported(inst); + if (ret) + goto error; + if (V4L2_TYPE_IS_OUTPUT(q->type)) ret = iris_vdec_streamon_input(inst); else if (V4L2_TYPE_IS_CAPTURE(q->type)) From patchwork Tue Aug 27 10:05:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 824182 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33A0A1A2C1D; Tue, 27 Aug 2024 10:06:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753197; cv=none; b=oc1HkteEYfHwiLNJs3nSlBqOdBiVrMD2exLjGhbamhuFGcNgeiGRqT4ac6dOiHBasAQl6d6HR2d0PJb51WOl3ZsGvmkwe9f+QDouRJRFyH4p0h4gDPQ/pr7+zQjl2VXH9UdeCOrKhvuj97PIT7KYeDZeRXzqjCZIsnmfFo+or1I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753197; c=relaxed/simple; bh=sRcEnTMGMEtOIyGmwRWTi2XXDHk0JS4ehbiTCWQc8H4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=VjfmbgcHWlRPWZffiNaNOAXgHb7Cuzc/4ZQ97xvfxxNKvvMG37eB7aoaIjW7Gvncn4biQ+UGfdcs3hnZTv/WUJCtWlbv6G2Jx+tNJqA75My3oIDa66LfXOiyVEbMWvxgwx++B4Tt7klbK2KvwdrbJ8oP3MOSs6z9e2pOMtbqJzQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=UI0HtRYV; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="UI0HtRYV" Received: by smtp.kernel.org (Postfix) with ESMTPS id D23F2C4FEF0; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753196; bh=sRcEnTMGMEtOIyGmwRWTi2XXDHk0JS4ehbiTCWQc8H4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=UI0HtRYV98DmWIuAbtMafLw0z5OHTdMT5AnWvZlPAGbEePm81oe5Lvp+TNpBlrVuN Ba4RNuFUQeFjL1+XQrJcxUuM49ULQiDml/eB0XO55ZM3ngW/Lc2DjfIJRPgogIu3MC JV8jhusLUAmqystZbKivNlqhOD9RdxReMrDHQH93pAActFC5FN/tMgZnfi5odDRllI b8FIaqxZq+c72nVLwlStlG46iJpYRLcaodmKUO0pbFgYv1EJcmsYmsQ6AHHuG7PIkR 07dZy2wFgmYQ10UWzUgfOhHij77Zdw2sw0nWCwDtOVdpGjgvyTIfclJbeOJB9tVndY SV87sMe56eXYA== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id C809EC54734; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:52 +0530 Subject: [PATCH v3 27/29] media: iris: implement power scaling for vpu2 and vpu3 Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-27-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal , Vedang Nagar X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753190; l=17651; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=ZMt5f7w2pDYigrH3LIYCwefJwvOPDMibnmpDxsQCOW4=; b=idJNBi9tdLUjcJ8SzR40vRA/coP1eGvxI1ootl9thRzzEMQstiuznQmafq7S0+NCNfUjHZY2f oKjwbV0oeTuCWuydd45TsBXeQpZTyhKJDGNLnAtnkgjkjnW3VQaB2Z+ X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Vedang Nagar Implement power scaling including vpu2 and vpu3 specific calculation for clock and bus bandwidth which depends on hardware configuration, codec format, resolution and frame rate. Signed-off-by: Vedang Nagar Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/iris_buffer.c | 3 + drivers/media/platform/qcom/iris/iris_instance.h | 6 + .../platform/qcom/iris/iris_platform_common.h | 21 +++ .../platform/qcom/iris/iris_platform_sm8250.c | 21 +++ .../platform/qcom/iris/iris_platform_sm8550.c | 27 ++++ drivers/media/platform/qcom/iris/iris_power.c | 149 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_power.h | 3 +- drivers/media/platform/qcom/iris/iris_vb2.c | 3 + drivers/media/platform/qcom/iris/iris_vdec.c | 7 + drivers/media/platform/qcom/iris/iris_vpu2.c | 28 ++++ drivers/media/platform/qcom/iris/iris_vpu3.c | 46 +++++++ drivers/media/platform/qcom/iris/iris_vpu_common.h | 1 + 12 files changed, 313 insertions(+), 2 deletions(-) diff --git a/drivers/media/platform/qcom/iris/iris_buffer.c b/drivers/media/platform/qcom/iris/iris_buffer.c index 3a11930a8076..8b874e0ad628 100644 --- a/drivers/media/platform/qcom/iris/iris_buffer.c +++ b/drivers/media/platform/qcom/iris/iris_buffer.c @@ -8,6 +8,7 @@ #include "iris_buffer.h" #include "iris_instance.h" +#include "iris_power.h" #include "iris_vpu_buffer.h" #define PIXELS_4K 4096 @@ -510,6 +511,8 @@ int iris_queue_deferred_buffers(struct iris_inst *inst, enum iris_buffer_type bu struct iris_buffer *buf = NULL; int ret; + iris_scale_power(inst); + if (buf_type == BUF_INPUT) { v4l2_m2m_for_each_src_buf_safe(m2m_ctx, buffer, n) { buf = to_iris_buffer(&buffer->vb); diff --git a/drivers/media/platform/qcom/iris/iris_instance.h b/drivers/media/platform/qcom/iris/iris_instance.h index dd1dad586a69..6ca16f855955 100644 --- a/drivers/media/platform/qcom/iris/iris_instance.h +++ b/drivers/media/platform/qcom/iris/iris_instance.h @@ -34,6 +34,9 @@ * @state: instance state * @sub_state: instance sub state * @once_per_session_set: boolean to set once per session property + * @max_input_data_size: max size of input data + * @power: structure of power info + * @icc_data: structure of interconnect data * @m2m_dev: a reference to m2m device structure * @m2m_ctx: a reference to m2m context structure * @subscriptions: variable to hold current events subscriptions @@ -59,6 +62,9 @@ struct iris_inst { enum iris_inst_state state; enum iris_inst_sub_state sub_state; bool once_per_session_set; + size_t max_input_data_size; + struct iris_inst_power power; + struct icc_vote_data icc_data; struct v4l2_m2m_dev *m2m_dev; struct v4l2_m2m_ctx *m2m_ctx; unsigned int subscriptions; diff --git a/drivers/media/platform/qcom/iris/iris_platform_common.h b/drivers/media/platform/qcom/iris/iris_platform_common.h index 2abc96d56a4b..32d7b04181e2 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_common.h +++ b/drivers/media/platform/qcom/iris/iris_platform_common.h @@ -68,6 +68,10 @@ enum platform_inst_driver_cap_type { FRAME_WIDTH = 1, FRAME_HEIGHT, MBPF, + MB_CYCLES_VPP, + MB_CYCLES_VSP, + MB_CYCLES_FW, + MB_CYCLES_FW_VPP, NUM_COMV, INST_DRIVER_CAP_MAX, }; @@ -117,11 +121,26 @@ struct platform_inst_fw_cap { enum platform_inst_fw_cap_type cap_id); }; +struct bw_info { + u32 mbs_per_sec; + u32 bw_ddr; +}; + struct iris_core_power { u64 clk_freq; u64 icc_bw; }; +struct iris_inst_power { + u64 min_freq; + u32 icc_bw; +}; + +struct icc_vote_data { + int height, width; + u32 fps; +}; + enum platform_pm_domain_type { IRIS_CTRL_POWER_DOMAIN, IRIS_HW_POWER_DOMAIN, @@ -135,6 +154,8 @@ struct iris_platform_data { void (*set_preset_registers)(struct iris_core *core); const struct icc_info *icc_tbl; unsigned int icc_tbl_size; + const struct bw_info *bw_tbl_dec; + unsigned int bw_tbl_dec_size; const char * const *pmdomain_tbl; unsigned int pmdomain_tbl_size; const char * const *opp_pd_tbl; diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c index 81f8366bcdd2..5dcc39598853 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8250.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8250.c @@ -30,6 +30,18 @@ static struct platform_inst_driver_cap instance_driver_cap_data_sm8250[] = { .max = 138240, .value = 138240, }, + { + .cap_id = MB_CYCLES_VPP, + .min = 200, + .max = 200, + .value = 200, + }, + { + .cap_id = MB_CYCLES_VSP, + .min = 25, + .max = 25, + .value = 25, + }, }; static struct platform_inst_fw_cap instance_fw_cap_data_sm8250[] = { @@ -77,6 +89,13 @@ static const struct icc_info sm8250_icc_table[] = { static const char * const sm8250_clk_reset_table[] = { "bus", "core" }; +static const struct bw_info sm8250_bw_table_dec[] = { + { ((4096 * 2160) / 256) * 60, 2403000 }, + { ((4096 * 2160) / 256) * 30, 1224000 }, + { ((1920 * 1080) / 256) * 60, 812000 }, + { ((1920 * 1080) / 256) * 30, 416000 }, +}; + static const char * const sm8250_pmdomain_table[] = { "venus", "vcodec0" }; static const char * const sm8250_opp_pd_table[] = { "mx" }; @@ -124,6 +143,8 @@ struct iris_platform_data sm8250_data = { .icc_tbl_size = ARRAY_SIZE(sm8250_icc_table), .clk_rst_tbl = sm8250_clk_reset_table, .clk_rst_tbl_size = ARRAY_SIZE(sm8250_clk_reset_table), + .bw_tbl_dec = sm8250_bw_table_dec, + .bw_tbl_dec_size = ARRAY_SIZE(sm8250_bw_table_dec), .pmdomain_tbl = sm8250_pmdomain_table, .pmdomain_tbl_size = ARRAY_SIZE(sm8250_pmdomain_table), .opp_pd_tbl = sm8250_opp_pd_table, diff --git a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c index 1b7644fdd21b..1a1e112f3e97 100644 --- a/drivers/media/platform/qcom/iris/iris_platform_sm8550.c +++ b/drivers/media/platform/qcom/iris/iris_platform_sm8550.c @@ -32,6 +32,24 @@ static struct platform_inst_driver_cap instance_driver_cap_data_sm8550[] = { .max = 138240, .value = 138240, }, + { + .cap_id = MB_CYCLES_VPP, + .min = 200, + .max = 200, + .value = 200, + }, + { + .cap_id = MB_CYCLES_FW, + .min = 489583, + .max = 489583, + .value = 489583, + }, + { + .cap_id = MB_CYCLES_FW_VPP, + .min = 66234, + .max = 66234, + .value = 66234, + }, { .cap_id = NUM_COMV, .min = 0, @@ -170,6 +188,13 @@ static const struct icc_info sm8550_icc_table[] = { static const char * const sm8550_clk_reset_table[] = { "bus" }; +static const struct bw_info sm8550_bw_table_dec[] = { + { ((4096 * 2160) / 256) * 60, 1608000 }, + { ((4096 * 2160) / 256) * 30, 826000 }, + { ((1920 * 1080) / 256) * 60, 567000 }, + { ((1920 * 1080) / 256) * 30, 294000 }, +}; + static const char * const sm8550_pmdomain_table[] = { "venus", "vcodec0" }; static const char * const sm8550_opp_pd_table[] = { "mxc", "mmcx" }; @@ -243,6 +268,8 @@ struct iris_platform_data sm8550_data = { .icc_tbl_size = ARRAY_SIZE(sm8550_icc_table), .clk_rst_tbl = sm8550_clk_reset_table, .clk_rst_tbl_size = ARRAY_SIZE(sm8550_clk_reset_table), + .bw_tbl_dec = sm8550_bw_table_dec, + .bw_tbl_dec_size = ARRAY_SIZE(sm8550_bw_table_dec), .pmdomain_tbl = sm8550_pmdomain_table, .pmdomain_tbl_size = ARRAY_SIZE(sm8550_pmdomain_table), .opp_pd_tbl = sm8550_opp_pd_table, diff --git a/drivers/media/platform/qcom/iris/iris_power.c b/drivers/media/platform/qcom/iris/iris_power.c index e697c27c8a8a..276637a48df5 100644 --- a/drivers/media/platform/qcom/iris/iris_power.c +++ b/drivers/media/platform/qcom/iris/iris_power.c @@ -3,10 +3,157 @@ * Copyright (c) 2022-2024 Qualcomm Innovation Center, Inc. All rights reserved. */ -#include "iris_core.h" +#include +#include + +#include "iris_buffer.h" +#include "iris_instance.h" #include "iris_power.h" +#include "iris_resources.h" #include "iris_vpu_common.h" +static u32 iris_calc_bw(struct iris_inst *inst, struct icc_vote_data *data) +{ + const struct bw_info *bw_tbl = NULL; + struct iris_core *core = inst->core; + unsigned int num_rows = 0; + unsigned int i, mbs, mbps; + u32 icc_bw = 0; + + if (!data) + goto exit; + + mbs = DIV_ROUND_UP(data->height, 16) * DIV_ROUND_UP(data->width, 16); + mbps = mbs * data->fps; + if (mbps == 0) + goto exit; + + bw_tbl = core->iris_platform_data->bw_tbl_dec; + num_rows = core->iris_platform_data->bw_tbl_dec_size; + + if (!bw_tbl || num_rows == 0) + goto exit; + + for (i = 0; i < num_rows; i++) { + if (i != 0 && mbps > bw_tbl[i].mbs_per_sec) + break; + + icc_bw = bw_tbl[i].bw_ddr; + } + +exit: + return icc_bw; +} + +static int iris_set_interconnects(struct iris_inst *inst) +{ + struct iris_core *core = inst->core; + struct iris_inst *instance; + u64 total_bw_ddr = 0; + int ret; + + mutex_lock(&core->lock); + list_for_each_entry(instance, &core->instances, list) { + if (!instance->max_input_data_size) + continue; + + total_bw_ddr += instance->power.icc_bw; + } + + ret = iris_set_icc_bw(core, total_bw_ddr); + + mutex_unlock(&core->lock); + + return ret; +} + +static int iris_vote_interconnects(struct iris_inst *inst) +{ + struct v4l2_format *inp_f; + struct icc_vote_data *vote_data; + + vote_data = &inst->icc_data; + + inp_f = inst->fmt_src; + + vote_data->width = inp_f->fmt.pix_mp.width; + vote_data->height = inp_f->fmt.pix_mp.height; + vote_data->fps = DEFAULT_FPS; + + inst->power.icc_bw = iris_calc_bw(inst, vote_data); + + return iris_set_interconnects(inst); +} + +static int iris_set_clocks(struct iris_inst *inst) +{ + struct iris_core *core = inst->core; + struct iris_inst *instance; + int ret = 0; + u64 freq; + + mutex_lock(&core->lock); + + freq = 0; + list_for_each_entry(instance, &core->instances, list) { + if (!instance->max_input_data_size) + continue; + + freq += instance->power.min_freq; + } + + core->power.clk_freq = freq; + + ret = iris_opp_set_rate(core, freq); + + mutex_unlock(&core->lock); + + return ret; +} + +static int iris_scale_clocks(struct iris_inst *inst) +{ + const struct vpu_ops *vpu_ops = inst->core->iris_platform_data->vpu_ops; + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + struct v4l2_m2m_buffer *buffer, *n; + struct iris_buffer *buf = NULL; + size_t data_size = 0; + + v4l2_m2m_for_each_src_buf_safe(m2m_ctx, buffer, n) { + buf = to_iris_buffer(&buffer->vb); + data_size = max(data_size, buf->data_size); + } + + inst->max_input_data_size = data_size; + + if (!inst->max_input_data_size) + return 0; + + inst->power.min_freq = vpu_ops->calc_freq(inst, inst->max_input_data_size); + + return iris_set_clocks(inst); +} + +int iris_scale_power(struct iris_inst *inst) +{ + struct iris_core *core = inst->core; + int ret; + + if (pm_runtime_suspended(core->dev)) { + ret = pm_runtime_resume_and_get(core->dev); + if (ret < 0) + return ret; + + pm_runtime_put_autosuspend(core->dev); + } + + ret = iris_scale_clocks(inst); + if (ret) + return ret; + + return iris_vote_interconnects(inst); +} + void iris_power_off(struct iris_core *core) { if (!core->power_enabled) diff --git a/drivers/media/platform/qcom/iris/iris_power.h b/drivers/media/platform/qcom/iris/iris_power.h index ff9b6be3b086..0a7ce1772893 100644 --- a/drivers/media/platform/qcom/iris/iris_power.h +++ b/drivers/media/platform/qcom/iris/iris_power.h @@ -6,8 +6,9 @@ #ifndef _IRIS_POWER_H_ #define _IRIS_POWER_H_ -struct iris_core; +struct iris_inst; +int iris_scale_power(struct iris_inst *inst); int iris_power_on(struct iris_core *core); void iris_power_off(struct iris_core *core); diff --git a/drivers/media/platform/qcom/iris/iris_vb2.c b/drivers/media/platform/qcom/iris/iris_vb2.c index 9dfbd0d66b61..974fce49d8fb 100644 --- a/drivers/media/platform/qcom/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/iris/iris_vb2.c @@ -11,6 +11,7 @@ #include "iris_instance.h" #include "iris_vb2.h" #include "iris_vdec.h" +#include "iris_power.h" #include "iris_vpu_buffer.h" static int iris_check_core_mbpf(struct iris_inst *inst) @@ -220,6 +221,8 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count) goto error; } + iris_scale_power(inst); + ret = iris_check_session_supported(inst); if (ret) goto error; diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index b8d2d6846daa..4b8670612dab 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -9,6 +9,7 @@ #include "iris_buffer.h" #include "iris_ctrls.h" #include "iris_instance.h" +#include "iris_power.h" #include "iris_vdec.h" #include "iris_vpu_buffer.h" @@ -388,6 +389,8 @@ static int iris_vdec_process_streamon_input(struct iris_inst *inst) enum iris_inst_sub_state set_sub_state = IRIS_INST_SUB_NONE; int ret; + iris_scale_power(inst); + ret = hfi_ops->session_start(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); if (ret) return ret; @@ -461,6 +464,8 @@ static int iris_vdec_process_streamon_output(struct iris_inst *inst) bool drain_active = false, drc_active = false; int ret; + iris_scale_power(inst); + drain_active = inst->sub_state & IRIS_INST_SUB_DRAIN && inst->sub_state & IRIS_INST_SUB_DRAIN_LAST; @@ -598,6 +603,8 @@ int iris_vdec_qbuf(struct iris_inst *inst, struct vb2_v4l2_buffer *vbuf) if (ret) return ret; + iris_scale_power(inst); + return iris_queue_buffer(inst, buf); } diff --git a/drivers/media/platform/qcom/iris/iris_vpu2.c b/drivers/media/platform/qcom/iris/iris_vpu2.c index bd8427411576..2e0777a3bf68 100644 --- a/drivers/media/platform/qcom/iris/iris_vpu2.c +++ b/drivers/media/platform/qcom/iris/iris_vpu2.c @@ -6,6 +6,34 @@ #include "iris_instance.h" #include "iris_vpu_common.h" +static u64 iris_vpu2_calc_freq(struct iris_inst *inst, size_t data_size) +{ + unsigned long vpp_freq = 0, vsp_freq = 0; + u32 fps, mbpf, height = 0, width = 0; + struct v4l2_format *inp_f; + u32 mbs_per_second; + + inp_f = inst->fmt_src; + width = max(inp_f->fmt.pix_mp.width, inst->crop.width); + height = max(inp_f->fmt.pix_mp.height, inst->crop.height); + + mbpf = NUM_MBS_PER_FRAME(height, width); + fps = DEFAULT_FPS; + mbs_per_second = mbpf * fps; + + vpp_freq = mbs_per_second * inst->driver_cap[MB_CYCLES_VPP].value; + + /* 21 / 20 is overhead factor */ + vpp_freq += vpp_freq / 20; + vsp_freq = mbs_per_second * inst->driver_cap[MB_CYCLES_VSP].value; + + /* 10 / 7 is overhead factor */ + vsp_freq += ((fps * data_size * 8) * 10) / 7; + + return max(vpp_freq, vsp_freq); +} + const struct vpu_ops iris_vpu2_ops = { .power_off_hw = iris_vpu_power_off_hw, + .calc_freq = iris_vpu2_calc_freq, }; diff --git a/drivers/media/platform/qcom/iris/iris_vpu3.c b/drivers/media/platform/qcom/iris/iris_vpu3.c index 414b8ec67b33..8cff853044aa 100644 --- a/drivers/media/platform/qcom/iris/iris_vpu3.c +++ b/drivers/media/platform/qcom/iris/iris_vpu3.c @@ -74,6 +74,52 @@ static void iris_vpu3_power_off_hardware(struct iris_core *core) iris_vpu_power_off_hw(core); } +static u64 iris_vpu3_calculate_frequency(struct iris_inst *inst, size_t data_size) +{ + u64 vsp_cycles = 0, vpp_cycles = 0, fw_cycles = 0; + u64 fw_vpp_cycles = 0, bitrate = 0, freq = 0; + u32 base_cycles = 0, fps, mbpf; + u32 height = 0, width = 0; + struct v4l2_format *inp_f; + u32 mbs_per_second; + + inp_f = inst->fmt_src; + width = max(inp_f->fmt.pix_mp.width, inst->crop.width); + height = max(inp_f->fmt.pix_mp.height, inst->crop.height); + + mbpf = NUM_MBS_PER_FRAME(height, width); + fps = DEFAULT_FPS; + mbs_per_second = mbpf * fps; + + fw_cycles = fps * inst->driver_cap[MB_CYCLES_FW].value; + fw_vpp_cycles = fps * inst->driver_cap[MB_CYCLES_FW_VPP].value; + + vpp_cycles = mbs_per_second * inst->driver_cap[MB_CYCLES_VPP].value / + inst->fw_cap[PIPE].value; + vpp_cycles += max(vpp_cycles / 20, fw_vpp_cycles); + + if (inst->fw_cap[PIPE].value > 1) + vpp_cycles += div_u64(vpp_cycles * 59, 1000); + + bitrate = fps * data_size * 8; + vsp_cycles = bitrate; + + base_cycles = 0; + vsp_cycles = div_u64(vsp_cycles, 2); + + vsp_cycles = div_u64(vsp_cycles * 21, 20); + + if (inst->fw_cap[STAGE].value == STAGE_1) + vsp_cycles = vsp_cycles * 3; + + vsp_cycles += mbs_per_second * base_cycles; + + freq = max3(vpp_cycles, vsp_cycles, fw_cycles); + + return freq; +} + const struct vpu_ops iris_vpu3_ops = { .power_off_hw = iris_vpu3_power_off_hardware, + .calc_freq = iris_vpu3_calculate_frequency, }; diff --git a/drivers/media/platform/qcom/iris/iris_vpu_common.h b/drivers/media/platform/qcom/iris/iris_vpu_common.h index 525bbb52dd79..fce17351ed67 100644 --- a/drivers/media/platform/qcom/iris/iris_vpu_common.h +++ b/drivers/media/platform/qcom/iris/iris_vpu_common.h @@ -13,6 +13,7 @@ extern const struct vpu_ops iris_vpu3_ops; struct vpu_ops { void (*power_off_hw)(struct iris_core *core); + u64 (*calc_freq)(struct iris_inst *inst, size_t data_size); }; int iris_vpu_boot_firmware(struct iris_core *core); From patchwork Tue Aug 27 10:05:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 822717 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 35C561A2C20; Tue, 27 Aug 2024 10:06:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753197; cv=none; b=U7wnLadU650RtPwAbCJe/62ZSRV5uFKZdP670M/rMFKdth6PDBOPGfeS+pnDYA9+N8c1URV/JCLMMUORTgbhOPvnqjOZNN49pVJTkBzp14RsAHrxGj53O50So1FA3HqcMxe0ALAJiSZIJWDyaI2KiRmCwiF9iTeOKSg2vgCSs+E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753197; c=relaxed/simple; bh=GoMTyeXcFceWMyNEb7SzalKpYrXcDS9NOTzgihbf2BQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=pGs/tcq2z6j6HlqbN8vd5YlxxXP5fjjh4UVA21LwxU9jDK5A/XIThHBQ6EiRr9utSR1stCIZPbwpX5nCUKCzJ6gTdGiMa0w8Uiu6WK3GcAXG85KmDWpvThdZZuxAXXtPV08ocxtxOw4lXjms7UR/Ww0xpkc5nyjgMcMUDbg2D3k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ajXxJoOw; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ajXxJoOw" Received: by smtp.kernel.org (Postfix) with ESMTPS id E9C6CC4DDEB; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753197; bh=GoMTyeXcFceWMyNEb7SzalKpYrXcDS9NOTzgihbf2BQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=ajXxJoOwqplTxWHR7Vd/EO32CVpTdJM1QtX0F6tfxULo8//MPLU/K3QjyPX/3vRFw 4TJAo/GlEXyNCfnD5ucN1cRmNKBRWMo5pNdcoeP8A8V4ikxMm5s/1xtQ5dfttfSSBy Jb26fZuQyoqgJF2P7fAr+2IrUDh/7KTOUqi3R1yZf7GEPFtr0e60JSOVVZ8D7Y8ZRf GocXIzJOsLGkEzGLffOscvDeUBTV23owU6wK1F+OzEsNFhbSpbeoYYH4nbtcLHeSyi 33w8cRQw3a0KsdmJDFEUhhpoxONaq+BQC/H4NYlTgxj12rzXOH/SFwaMKSluv+V9yt W4hH5xyFSQaDg== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id DE724C54739; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:53 +0530 Subject: [PATCH v3 28/29] media: iris: add allow checks for v4l2 ioctls Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-28-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal , Vedang Nagar X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753191; l=8955; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=mbg1x1FKjtUUbKPnVop/IotY6F1Q5WWGZ5Q6GZ+U/H4=; b=x9ID+lTi2aLIf+hkvK5acH//RdXm0jMkr2fOcFFEe37OitKmzB/C4Xd7qFiE6E/PWeV/KsiCK MXqSDRYjbSKCHrejtoawLNGEsXWEWemi9CSYn1kW1+sezuOodr6/x7u X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Vedang Nagar Based on instance state machine, add allow checks for all the v4l2 and vb2 ops. Signed-off-by: Vedang Nagar Signed-off-by: Dikshita Agarwal --- drivers/media/platform/qcom/iris/iris_ctrls.c | 3 + drivers/media/platform/qcom/iris/iris_state.c | 80 +++++++++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_state.h | 10 ++++ drivers/media/platform/qcom/iris/iris_vb2.c | 14 +++++ drivers/media/platform/qcom/iris/iris_vdec.c | 5 ++ drivers/media/platform/qcom/iris/iris_vidc.c | 15 +++++ 6 files changed, 127 insertions(+) diff --git a/drivers/media/platform/qcom/iris/iris_ctrls.c b/drivers/media/platform/qcom/iris/iris_ctrls.c index 64b2f19c2b03..5aeb48a8ccb6 100644 --- a/drivers/media/platform/qcom/iris/iris_ctrls.c +++ b/drivers/media/platform/qcom/iris/iris_ctrls.c @@ -81,6 +81,9 @@ static int iris_vdec_op_s_ctrl(struct v4l2_ctrl *ctrl) if (!iris_valid_cap_id(cap_id)) return -EINVAL; + if (!iris_allow_s_ctrl(inst, cap_id)) + return -EBUSY; + cap[cap_id].flags |= CAP_FLAG_CLIENT_SET; inst->fw_cap[cap_id].value = ctrl->val; diff --git a/drivers/media/platform/qcom/iris/iris_state.c b/drivers/media/platform/qcom/iris/iris_state.c index d37f99d72491..1c7437bc808b 100644 --- a/drivers/media/platform/qcom/iris/iris_state.c +++ b/drivers/media/platform/qcom/iris/iris_state.c @@ -77,6 +77,56 @@ int iris_inst_change_state(struct iris_inst *inst, return 0; } +bool iris_allow_s_fmt(struct iris_inst *inst, u32 type) +{ + return (inst->state == IRIS_INST_DEINIT) || + (inst->state == IRIS_INST_INIT) || + (V4L2_TYPE_IS_CAPTURE(type) && inst->state == IRIS_INST_INPUT_STREAMING) || + (V4L2_TYPE_IS_OUTPUT(type) && inst->state == IRIS_INST_OUTPUT_STREAMING); +} + +bool iris_allow_reqbufs(struct iris_inst *inst, u32 type) +{ + return (inst->state == IRIS_INST_DEINIT) || + (inst->state == IRIS_INST_INIT) || + (V4L2_TYPE_IS_CAPTURE(type) && inst->state == IRIS_INST_INPUT_STREAMING) || + (V4L2_TYPE_IS_OUTPUT(type) && inst->state == IRIS_INST_OUTPUT_STREAMING); +} + +bool iris_allow_qbuf(struct iris_inst *inst, u32 type) +{ + return (V4L2_TYPE_IS_OUTPUT(type) && inst->state == IRIS_INST_INPUT_STREAMING) || + (V4L2_TYPE_IS_OUTPUT(type) && inst->state == IRIS_INST_STREAMING) || + (V4L2_TYPE_IS_CAPTURE(type) && inst->state == IRIS_INST_OUTPUT_STREAMING) || + (V4L2_TYPE_IS_CAPTURE(type) && inst->state == IRIS_INST_STREAMING); +} + +bool iris_allow_streamon(struct iris_inst *inst, u32 type) +{ + return (V4L2_TYPE_IS_OUTPUT(type) && inst->state == IRIS_INST_INIT) || + (V4L2_TYPE_IS_OUTPUT(type) && inst->state == IRIS_INST_OUTPUT_STREAMING) || + (V4L2_TYPE_IS_CAPTURE(type) && inst->state == IRIS_INST_INIT) || + (V4L2_TYPE_IS_CAPTURE(type) && inst->state == IRIS_INST_INPUT_STREAMING); +} + +bool iris_allow_streamoff(struct iris_inst *inst, u32 type) +{ + return (V4L2_TYPE_IS_OUTPUT(type) && inst->state == IRIS_INST_INPUT_STREAMING) || + (V4L2_TYPE_IS_OUTPUT(type) && inst->state == IRIS_INST_STREAMING) || + (V4L2_TYPE_IS_CAPTURE(type) && inst->state == IRIS_INST_OUTPUT_STREAMING) || + (V4L2_TYPE_IS_CAPTURE(type) && inst->state == IRIS_INST_STREAMING) || + inst->state == IRIS_INST_ERROR; +} + +bool iris_allow_s_ctrl(struct iris_inst *inst, u32 cap_id) +{ + return ((inst->state == IRIS_INST_DEINIT) || + (inst->state == IRIS_INST_INIT) || + ((inst->fw_cap[cap_id].flags & CAP_FLAG_DYNAMIC_ALLOWED) && + (inst->state == IRIS_INST_INPUT_STREAMING || + inst->state == IRIS_INST_STREAMING))); +} + int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane) { enum iris_inst_state new_state = IRIS_INST_ERROR; @@ -214,3 +264,33 @@ int iris_inst_sub_state_change_pause(struct iris_inst *inst, u32 plane) return iris_inst_change_sub_state(inst, 0, set_sub_state); } + +static inline bool iris_drc_pending(struct iris_inst *inst) +{ + return inst->sub_state & IRIS_INST_SUB_DRC && + inst->sub_state & IRIS_INST_SUB_DRC_LAST; +} + +static inline bool iris_drain_pending(struct iris_inst *inst) +{ + return inst->sub_state & IRIS_INST_SUB_DRAIN && + inst->sub_state & IRIS_INST_SUB_DRAIN_LAST; +} + +bool iris_allow_cmd(struct iris_inst *inst, u32 cmd) +{ + if (cmd == V4L2_DEC_CMD_START) { + if (inst->state == IRIS_INST_INPUT_STREAMING || + inst->state == IRIS_INST_OUTPUT_STREAMING || + inst->state == IRIS_INST_STREAMING) + if (iris_drc_pending(inst) || iris_drain_pending(inst)) + return true; + } else if (cmd == V4L2_DEC_CMD_STOP) { + if (inst->state == IRIS_INST_INPUT_STREAMING || + inst->state == IRIS_INST_STREAMING) + if (inst->sub_state != IRIS_INST_SUB_DRAIN) + return true; + } + + return false; +} diff --git a/drivers/media/platform/qcom/iris/iris_state.h b/drivers/media/platform/qcom/iris/iris_state.h index a01037918a9c..9516425fcfa7 100644 --- a/drivers/media/platform/qcom/iris/iris_state.h +++ b/drivers/media/platform/qcom/iris/iris_state.h @@ -133,10 +133,20 @@ int iris_inst_change_state(struct iris_inst *inst, int iris_inst_change_sub_state(struct iris_inst *inst, enum iris_inst_sub_state clear_sub_state, enum iris_inst_sub_state set_sub_state); + +bool iris_allow_s_fmt(struct iris_inst *inst, u32 type); +bool iris_allow_reqbufs(struct iris_inst *inst, u32 type); +bool iris_allow_qbuf(struct iris_inst *inst, u32 type); +bool iris_allow_streamon(struct iris_inst *inst, u32 type); +bool iris_allow_streamoff(struct iris_inst *inst, u32 type); +bool iris_allow_s_ctrl(struct iris_inst *inst, u32 cap_id); + int iris_inst_state_change_streamon(struct iris_inst *inst, u32 plane); int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane); int iris_inst_sub_state_change_drc(struct iris_inst *inst); int iris_inst_sub_state_change_drain_last(struct iris_inst *inst); int iris_inst_sub_state_change_drc_last(struct iris_inst *inst); int iris_inst_sub_state_change_pause(struct iris_inst *inst, u32 plane); +bool iris_allow_cmd(struct iris_inst *inst, u32 cmd); + #endif diff --git a/drivers/media/platform/qcom/iris/iris_vb2.c b/drivers/media/platform/qcom/iris/iris_vb2.c index 974fce49d8fb..73c274844788 100644 --- a/drivers/media/platform/qcom/iris/iris_vb2.c +++ b/drivers/media/platform/qcom/iris/iris_vb2.c @@ -120,6 +120,11 @@ int iris_vb2_queue_setup(struct vb2_queue *q, goto unlock; } + if (!iris_allow_reqbufs(inst, q->type)) { + ret = -EBUSY; + goto unlock; + } + core = inst->core; f = V4L2_TYPE_IS_OUTPUT(q->type) ? inst->fmt_src : inst->fmt_dst; @@ -215,6 +220,11 @@ int iris_vb2_start_streaming(struct vb2_queue *q, unsigned int count) goto error; } + if (!iris_allow_streamon(inst, q->type)) { + ret = -EBUSY; + goto error; + } + if (!V4L2_TYPE_IS_OUTPUT(q->type) && !V4L2_TYPE_IS_CAPTURE(q->type)) { ret = -EINVAL; @@ -269,6 +279,10 @@ void iris_vb2_stop_streaming(struct vb2_queue *q) return; mutex_lock(&inst->lock); + if (!iris_allow_streamoff(inst, q->type)) { + ret = -EBUSY; + goto exit; + } if (!V4L2_TYPE_IS_OUTPUT(q->type) && !V4L2_TYPE_IS_CAPTURE(q->type)) diff --git a/drivers/media/platform/qcom/iris/iris_vdec.c b/drivers/media/platform/qcom/iris/iris_vdec.c index 4b8670612dab..dfe7fedc234f 100644 --- a/drivers/media/platform/qcom/iris/iris_vdec.c +++ b/drivers/media/platform/qcom/iris/iris_vdec.c @@ -603,6 +603,11 @@ int iris_vdec_qbuf(struct iris_inst *inst, struct vb2_v4l2_buffer *vbuf) if (ret) return ret; + if (!iris_allow_qbuf(inst, vb2->type)) { + buf->attr |= BUF_ATTR_DEFERRED; + return 0; + } + iris_scale_power(inst); return iris_queue_buffer(inst, buf); diff --git a/drivers/media/platform/qcom/iris/iris_vidc.c b/drivers/media/platform/qcom/iris/iris_vidc.c index 07eb276472f6..a22b4c1226df 100644 --- a/drivers/media/platform/qcom/iris/iris_vidc.c +++ b/drivers/media/platform/qcom/iris/iris_vidc.c @@ -316,6 +316,11 @@ static int iris_try_fmt_vid_mplane(struct file *filp, void *fh, struct v4l2_form goto unlock; } + if (!iris_allow_s_fmt(inst, f->type)) { + ret = -EBUSY; + goto unlock; + } + ret = iris_vdec_try_fmt(inst, f); unlock: @@ -339,6 +344,11 @@ static int iris_s_fmt_vid_mplane(struct file *filp, void *fh, struct v4l2_format goto unlock; } + if (!iris_allow_s_fmt(inst, f->type)) { + ret = -EBUSY; + goto unlock; + } + ret = iris_vdec_s_fmt(inst, f); unlock: @@ -624,6 +634,11 @@ static int iris_dec_cmd(struct file *filp, void *fh, if (inst->state == IRIS_INST_DEINIT) goto unlock; + if (!iris_allow_cmd(inst, dec->cmd)) { + ret = -EBUSY; + goto unlock; + } + if (dec->cmd == V4L2_DEC_CMD_START) ret = iris_vdec_start_cmd(inst); else if (dec->cmd == V4L2_DEC_CMD_STOP) From patchwork Tue Aug 27 10:05:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dikshita Agarwal via B4 Relay X-Patchwork-Id: 822716 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3D2861A2C2B; Tue, 27 Aug 2024 10:06:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753197; cv=none; b=RjDVphGkBsJyx1mQZULTyhw4UDPhgxzTxYR863XxAceKcnQ3UDUakypAjY0rbls+rUoxf61/NOwNAKzdEyy18hKnEiA79gtqyZ1tPiVWSzihtOQYFZLQF9G/wIN6VLhzeB7Y0/h5qrYqRNOQBCctp54AGifAHOzp5S0J7mCsaDs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1724753197; c=relaxed/simple; bh=Dlq7epyisEHueyh2W/kuxJ5NQRtjMjbGbMDDIMhUmK8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=D8Wtx8QqiiD2ev1DK/487Su7tpC+SbvMuS/nHA/QFApc+WJCXQ0rGsHZtYLbFYrWgbntJYZPDR+sHsbNIwAs4gpWIGObaorfkqquOEJ0Gz4ssc/ETzd7l0fePKP4bzfe64S+SFIidxTPHjR1SjvkP3j5p6HgyIFwWM4af7RXFLQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=IKYuc+Al; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="IKYuc+Al" Received: by smtp.kernel.org (Postfix) with ESMTPS id 0CA3BC4FEF7; Tue, 27 Aug 2024 10:06:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1724753197; bh=Dlq7epyisEHueyh2W/kuxJ5NQRtjMjbGbMDDIMhUmK8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:Reply-To:From; b=IKYuc+Aljoyq3QSogG6oeXww3mIyhyh4QDsZlCl6Xri9k92XrtI/XShZzpl5T52ZG QeG/9tKrY7Hg6OFPehfLkxd8lBKobFTsI0s7UqGQgPhnXTbhc0BwQySLf9s4zRJXo/ fwArMsH4YU8l8ETuXknjWAq4/cdIegXuOPx6yAyar1DVbF3R6yEaQ2L43lmwJhRGX6 31fdbOYUG4fdSDo0T+bmYNCGB9gmi2sbPsswLfswED91szPFTrOl05n5D+Hn6WukVs X20j5/KXgdxeAKzWOqveIgSgm47RZXt+El/DNMJdNZI4Djp8qLa6GtTmHd3lXuoGa8 YPO8++qveRX3A== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id F37FEC54735; Tue, 27 Aug 2024 10:06:36 +0000 (UTC) From: Dikshita Agarwal via B4 Relay Date: Tue, 27 Aug 2024 15:35:54 +0530 Subject: [PATCH v3 29/29] media: iris: add check to allow sub states transitions Precedence: bulk X-Mailing-List: linux-arm-msm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20240827-iris_v3-v3-29-c5fdbbe65e70@quicinc.com> References: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> In-Reply-To: <20240827-iris_v3-v3-0-c5fdbbe65e70@quicinc.com> To: Vikash Garodia , Abhinav Kumar , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Philipp Zabel Cc: linux-media@vger.kernel.org, linux-arm-msm@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, Dikshita Agarwal , Vedang Nagar X-Mailer: b4 0.15-dev X-Developer-Signature: v=1; a=ed25519-sha256; t=1724753191; l=4597; i=quic_dikshita@quicinc.com; s=20240826; h=from:subject:message-id; bh=2EGrcsprZLpn6qAzmt8V0SRE4y1luktgGIACm9k/2VE=; b=7+MuEKA+zwz6gaIbmSjqsqr4G/QiPXTFv4WrtXRxQay//0VRqtWaLeJ4SSxndZZhdVbnsVHHz QwATFuYhcllAugnP9VBcyWrTgH7G4fJP4AwyCALfQqXRjnztrPj3L5h X-Developer-Key: i=quic_dikshita@quicinc.com; a=ed25519; pk=+c7562uu1Y968VTv9z59ch2v3jmlO2Qv3uX7srN3LJY= X-Endpoint-Received: by B4 Relay for quic_dikshita@quicinc.com/20240826 with auth_id=199 X-Original-From: Dikshita Agarwal Reply-To: quic_dikshita@quicinc.com From: Vedang Nagar Based on state machine design, add allow checks to transition from one sub-state to another sub-states. Signed-off-by: Vedang Nagar Signed-off-by: Dikshita Agarwal --- .../platform/qcom/iris/iris_hfi_gen1_command.c | 12 ++++++- drivers/media/platform/qcom/iris/iris_state.c | 42 ++++++++++++++++++++++ drivers/media/platform/qcom/iris/iris_state.h | 1 + 3 files changed, 54 insertions(+), 1 deletion(-) diff --git a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c index 17333768afa2..2f3e40368cce 100644 --- a/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +++ b/drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c @@ -135,6 +135,9 @@ static int iris_hfi_gen1_session_start(struct iris_inst *inst, u32 plane) if (!V4L2_TYPE_IS_OUTPUT(plane)) return 0; + if (inst->sub_state & IRIS_INST_SUB_LOAD_RESOURCES) + return 0; + reinit_completion(&inst->completion); iris_hfi_gen1_packet_session_cmd(inst, &packet, HFI_CMD_SESSION_LOAD_RESOURCES); @@ -153,7 +156,11 @@ static int iris_hfi_gen1_session_start(struct iris_inst *inst, u32 plane) if (ret) return ret; - return iris_wait_for_session_response(inst, false); + ret = iris_wait_for_session_response(inst, false); + if (ret) + return ret; + + return iris_inst_change_sub_state(inst, 0, IRIS_INST_SUB_LOAD_RESOURCES); } static int iris_hfi_gen1_session_stop(struct iris_inst *inst, u32 plane) @@ -180,6 +187,9 @@ static int iris_hfi_gen1_session_stop(struct iris_inst *inst, u32 plane) ret = iris_hfi_queue_cmd_write(core, &pkt, pkt.shdr.hdr.size); if (!ret) ret = iris_wait_for_session_response(inst, false); + + iris_inst_change_sub_state(inst, IRIS_INST_SUB_LOAD_RESOURCES, 0); + iris_helper_buffers_done(inst, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE, VB2_BUF_STATE_ERROR); iris_helper_buffers_done(inst, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE, diff --git a/drivers/media/platform/qcom/iris/iris_state.c b/drivers/media/platform/qcom/iris/iris_state.c index 1c7437bc808b..ab7dc768ef25 100644 --- a/drivers/media/platform/qcom/iris/iris_state.c +++ b/drivers/media/platform/qcom/iris/iris_state.c @@ -165,6 +165,45 @@ int iris_inst_state_change_streamoff(struct iris_inst *inst, u32 plane) return iris_inst_change_state(inst, new_state); } +static int iris_inst_allow_sub_state(struct iris_inst *inst, enum iris_inst_sub_state sub_state) +{ + if (!sub_state) + return true; + + switch (inst->state) { + case IRIS_INST_INIT: + if (sub_state & IRIS_INST_SUB_LOAD_RESOURCES) + return true; + break; + case IRIS_INST_INPUT_STREAMING: + if (sub_state & (IRIS_INST_SUB_FIRST_IPSC | IRIS_INST_SUB_DRC | + IRIS_INST_SUB_DRAIN | IRIS_INST_SUB_INPUT_PAUSE)) + return true; + break; + case IRIS_INST_OUTPUT_STREAMING: + if (sub_state & (IRIS_INST_SUB_DRC_LAST | + IRIS_INST_SUB_DRAIN_LAST | IRIS_INST_SUB_OUTPUT_PAUSE)) + return true; + break; + case IRIS_INST_STREAMING: + if (sub_state & (IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRAIN | + IRIS_INST_SUB_DRC_LAST | IRIS_INST_SUB_DRAIN_LAST | + IRIS_INST_SUB_INPUT_PAUSE | IRIS_INST_SUB_OUTPUT_PAUSE)) + return true; + break; + case IRIS_INST_DEINIT: + if (sub_state & (IRIS_INST_SUB_DRC | IRIS_INST_SUB_DRAIN | + IRIS_INST_SUB_DRC_LAST | IRIS_INST_SUB_DRAIN_LAST | + IRIS_INST_SUB_INPUT_PAUSE | IRIS_INST_SUB_OUTPUT_PAUSE)) + return true; + break; + default: + return false; + } + + return false; +} + int iris_inst_change_sub_state(struct iris_inst *inst, enum iris_inst_sub_state clear_sub_state, enum iris_inst_sub_state set_sub_state) @@ -184,6 +223,9 @@ int iris_inst_change_sub_state(struct iris_inst *inst, prev_sub_state = inst->sub_state; + if (!iris_inst_allow_sub_state(inst, set_sub_state)) + return -EINVAL; + inst->sub_state |= set_sub_state; inst->sub_state &= ~clear_sub_state; diff --git a/drivers/media/platform/qcom/iris/iris_state.h b/drivers/media/platform/qcom/iris/iris_state.h index 9516425fcfa7..f60dc18ca30e 100644 --- a/drivers/media/platform/qcom/iris/iris_state.h +++ b/drivers/media/platform/qcom/iris/iris_state.h @@ -124,6 +124,7 @@ enum iris_inst_sub_state { IRIS_INST_SUB_DRAIN_LAST = BIT(4), IRIS_INST_SUB_INPUT_PAUSE = BIT(5), IRIS_INST_SUB_OUTPUT_PAUSE = BIT(6), + IRIS_INST_SUB_LOAD_RESOURCES = BIT(7), }; void iris_change_core_state(struct iris_core *core,