| Message ID | 20240615015734.1612108-1-detlev.casanova@collabora.com |
|---|---|
| Headers | show |
| Series | media: rockchip: Add rkvdec2 driver | expand |
Hi, Le dimanche 16 juin 2024 à 10:40 +0200, Diederik de Haas a écrit : > On Saturday, 15 June 2024 21:44:32 CEST Detlev Casanova wrote: > > > So is this just an (unfortunate) use of the same words or is that wiki > > > page just wrong ... or better yet: does rkvdec2 support RK356x too? > > > > Yes, the vdpu34x decoder on rk356x socs should be supported by this driver > > but I don't have boards to test that unfortunately. > > > > This might also be used on future rockchip releases like the rk3576. But > > they all have their own adaptations. If you can test it on rk3568 based > > hardware, I'll happily add a compatible for it. > > It would be great if you'd add a compatible for it. > I don't have rk3568 based HW, but I do have rk3566 based HW and AFAIK those > are the same when it comes to the 'video' stuff. Our usual approach is to require at least a "Test-by:" to include patches written without HW. I think it will come soon enough, and we can focus on getting the driver in at this moment. Nicolas
On Sunday, June 16, 2024 3:28:25 A.M. EDT Heiko Stuebner wrote: > Am Samstag, 15. Juni 2024, 21:55:54 CEST schrieb Detlev Casanova: > > On Saturday, June 15, 2024 4:25:27 A.M. EDT Jonas Karlman wrote: > > > Hi Detlev, > > > > > > On 2024-06-15 03:56, Detlev Casanova wrote: > > > > Add the rkvdec2 Video Decoder to the RK3588s devicetree. > > > > > > > > Signed-off-by: Detlev Casanova <detlev.casanova@collabora.com> > > > > --- > > > > > > > > .../boot/dts/rockchip/rk3588-rock-5b.dts | 4 ++++ > > > > .../boot/dts/rockchip/rk3588s-orangepi-5.dts | 4 ++++ > > > > arch/arm64/boot/dts/rockchip/rk3588s.dtsi | 19 > > > > +++++++++++++++++++ > > > > 3 files changed, 27 insertions(+) > > > > > > > > diff --git a/arch/arm64/boot/dts/rockchip/rk3588-rock-5b.dts > > > > b/arch/arm64/boot/dts/rockchip/rk3588-rock-5b.dts index > > > > c551b676860c..965322c24a65 100644 > > > > --- a/arch/arm64/boot/dts/rockchip/rk3588-rock-5b.dts > > > > +++ b/arch/arm64/boot/dts/rockchip/rk3588-rock-5b.dts > > > > @@ -503,6 +503,10 @@ &pwm1 { > > > > > > > > status = "okay"; > > > > > > > > }; > > > > > > > > +&rkvdec0 { > > > > + status = "okay"; > > > > +}; > > > > > > Enable of rkvdec0 should probably be split out from the patch that adds > > > the rkvdec0 node to soc dtsi. > > > > Ack > > > > > Also why is rkvdec0 only enabled on rock-5b and orangepi-5? > > > > I only could test on those two but I can enable it on all rk3588 devices. > > > > > > + > > > > > > > > &saradc { > > > > > > > > vref-supply = <&avcc_1v8_s0>; > > > > status = "okay"; > > > > > > > > diff --git a/arch/arm64/boot/dts/rockchip/rk3588s-orangepi-5.dts > > > > b/arch/arm64/boot/dts/rockchip/rk3588s-orangepi-5.dts index > > > > feea6b20a6bf..2828fb4c182a 100644 > > > > --- a/arch/arm64/boot/dts/rockchip/rk3588s-orangepi-5.dts > > > > +++ b/arch/arm64/boot/dts/rockchip/rk3588s-orangepi-5.dts > > > > @@ -321,6 +321,10 @@ typec5v_pwren: typec5v-pwren { > > > > > > > > }; > > > > > > > > }; > > > > > > > > +&rkvdec0 { > > > > + status = "okay"; > > > > +}; > > > > + > > > > > > > > &saradc { > > > > > > > > vref-supply = <&avcc_1v8_s0>; > > > > status = "okay"; > > > > > > > > diff --git a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi > > > > b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi index > > > > 0fecbf46e127..09672636dcea 100644 > > > > --- a/arch/arm64/boot/dts/rockchip/rk3588s.dtsi > > > > +++ b/arch/arm64/boot/dts/rockchip/rk3588s.dtsi > > > > @@ -3034,6 +3034,9 @@ system_sram2: sram@ff001000 { > > > > > > > > ranges = <0x0 0x0 0xff001000 0xef000>; > > > > #address-cells = <1>; > > > > #size-cells = <1>; > > > > > > Blank line is missing. > > > > > > > + rkvdec0_sram: rkvdec-sram@0 { > > > > + reg = <0x0 0x78000>; > > > > + }; > > > > > > > > }; > > > > > > > > pinctrl: pinctrl { > > > > > > > > @@ -3103,6 +3106,22 @@ gpio4: gpio@fec50000 { > > > > > > > > #interrupt-cells = <2>; > > > > > > > > }; > > > > > > > > }; > > > > > > > > + > > > > + rkvdec0: video-decoder@fdc38100 { > > > > + compatible = "rockchip,rk3588-vdec2"; > > > > + reg = <0x0 0xfdc38100 0x0 0x500>; > > > > + interrupts = <GIC_SPI 95 IRQ_TYPE_LEVEL_HIGH 0>; > > > > + clocks = <&cru ACLK_RKVDEC0>, <&cru HCLK_RKVDEC0>, > > > > <&cru > > > > > > CLK_RKVDEC0_CORE>, + <&cru > > > > CLK_RKVDEC0_CA>, <&cru > > > > > > CLK_RKVDEC0_HEVC_CA>; > > > > + clock-names = "axi", "ahb", "core", > > > > + "cabac", "hevc_cabac"; > > > > + assigned-clocks = <&cru ACLK_RKVDEC0>, <&cru > > > > CLK_RKVDEC0_CORE>, > > > > > > + <&cru CLK_RKVDEC0_CA>, <&cru > > > > CLK_RKVDEC0_HEVC_CA>; > > > > > > + assigned-clock-rates = <800000000>, <600000000>, > > > > + <600000000>, <1000000000>; > > > > + power-domains = <&power RK3588_PD_RKVDEC0>; > > > > > > iommus and resets should probably be added. > > > > > > > + status = "disabled"; > > > > + }; > > > > > > The iommu node for rkvdec0_mmu seem to be missing, is it not required to > > > be able to use memory >4GiB as decoding buffers? > > > > I need to check if the current rockchip iommu driver will work for this > > decoder. I remember that the iommu code for AV1 was a bit different, not > > sure about this rkvdec. > > > > > I would also consider adding the rkvdec1 node(s), if I am understanding > > > correctly they can both be used in a cluster or completely independent. > > > > They can be used independently, yes. I'll add rkvdec1 for rk3588 devices > > (rk3588s only has 1 core) > > Please check in with Sebastian Reichel about clusters/independent > controllers. He had a lot of fruitful discussions for the VEPU121/VPU121 > support he is working on. Yes, basically, it makes the driver check for all nodes with the same compatible and list the instances in the same device instance. If multicore is not supported, the other instances can simple be ignored. See here: https://lore.kernel.org/linux-kernel/20240613135034.31684-4-sebastian.reichel@collabora.com/ > Baseline being, while we want the hw to be described correctly wrt the > multiple instances, we don't generally want to expose them individually > to userspace, because that would then require userspace to do all the > scheduling. Yes, so Sebastien's patch will allow the device tree to have the 2 cores but only one visible by the userspace (even when multicore decoding is not supported)
Hi Piotr, On Wednesday, 29 January 2025 09:48:51 EST Piotr Oniszczuk wrote: > > Wiadomość napisana przez Detlev Casanova <detlev.casanova@collabora.com> w > > dniu 15 cze 2024, o godz. 21:44: > > > > > > > > > > Yes, the vdpu34x decoder on rk356x socs should be supported by this driver > > but I don't have boards to test that unfortunately. > > Detlev, > > Just FYI: > > I done some tests of rkvdec2 on 6.12.11 on 3588, 3568 and 3566 > > For enabling rkvdec2 on 356x i: > -add 356x compatible in rkvdec2.c > -add dtsi nodes like this: > https://github.com/warpme/minimyth2/blob/master/script/kernel/linux-6.12/fi > les/1078-arm64-dtsi-rockchip-rk356x-add-rkvdec2-video-decoder-nodes.patch > > With this i can say: > -on rk3588 i have some hevc 4k decoding perfectly but some others are > failing -on rk3566/3568 only subset of 3588’s samples is decoded well. (but > is works then works perfectly fine) -when not failing on 3588 sample fails > on 356x - is see errors like: > > [ 95.666669] iova: 0x00000000f2e80000 already mapped to 0x0000000037378000 > cannot remap to phys: 0x000000002f8c0000 prot: 0x3 [ 95.745082] iova: > 0x00000000f2f46000 already mapped to 0x00000000372b6000 cannot remap to > phys: 0x000000003a403000 prot: 0x3 [ 95.822012] iova: 0x00000000f2ee6000 > already mapped to 0x0000000037126000 cannot remap to phys: > 0x000000003a803000 prot: 0x3 [ 95.896802] iova: 0x00000000f2ec6000 > already mapped to 0x0000000029fe6000 cannot remap to phys: > 0x000000003a403000 prot: 0x3 turning-off iommu makes above errors disappear > - but sample still fails. I suppose you tested with my hevc branch, which is not really ready yet (Some ref frames will work but usually, it won't) Can you confirm which branch/commit you based your tests on ? For the iommu, do you see those errors like that only on 356x or also on 3588 ? The hevc branch should have the iommu patches to fix that kind of things. (but note that hevc support is really new, so it may have bugs with buffer allocations) > If anybody hints me for way/tool to analyse of playing/failing samples to > catch: what encoding specifics makes given sample failing to decode on > rkvdec2 - i'll be more that happy to provide details… (doing simple > mediainfo <file> shows no differences for me…) Few features are supported for HEVC as of now: - No scanlist support (only default 16x16 blocks will work) - Long term reference frames are also not configured yet. - hevc 10 bits is also not supported yet These are specific to the encoding and mediainfo won't really give you information on that, except maybe on the 10 bits format. You can also checkout YUView (https://github.com/IENT/YUView) to get information on media files structure, but I have had issues with HEVC support lately. Regards, Detlev.
> Wiadomość napisana przez Detlev Casanova <detlev.casanova@collabora.com> w dniu 29 sty 2025, o godz. 17:50: > > Hi Piotr, > > On Wednesday, 29 January 2025 09:48:51 EST Piotr Oniszczuk wrote: > > I suppose you tested with my hevc branch, which is not really ready yet (Some > ref frames will work but usually, it won't) Can you confirm which branch/commit > you based your tests on ? Indeed - i’m using your rkvdec2 hevc code from: https://gitlab.collabora.com/detlev/linux/-/commits/add-rkvdec2-driver-hevc with iommu from https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux/-/commit/bc47c445bfd9586115e9bcf5f231c5a5c5f0f828 and https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux/-/commit/9a9fd791513bc0d02c2242c88f23b41bd47de30a > > For the iommu, do you see those errors like that only on 356x or also on 3588 > ? 3588 has no any iommu errors. I see iommu errors only on 356x > The hevc branch should have the iommu patches to fix that kind of things. I’m using: https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux/-/commit/bc47c445bfd9586115e9bcf5f231c5a5c5f0f828 https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux/-/commit/9a9fd791513bc0d02c2242c88f23b41bd47de30a I see there is: https://gitlab.collabora.com/detlev/linux/-/commits/add-rkvdec2-driver-iommu Let me update iommu code to above branch and report back here how it goes in 3588 and 356x > > (but note that hevc support is really new, so it may have bugs with buffer > allocations) It is already great work. I’m awaiting for hevc in 35xx during last 2+ years :-) > >> If anybody hints me for way/tool to analyse of playing/failing samples to >> catch: what encoding specifics makes given sample failing to decode on >> rkvdec2 - i'll be more that happy to provide details… (doing simple >> mediainfo <file> shows no differences for me…) > > Few features are supported for HEVC as of now: > - No scanlist support (only default 16x16 blocks will work) > - Long term reference frames are also not configured yet. > - hevc 10 bits is also not supported yet > > These are specific to the encoding and mediainfo won't really give you > information on that, except maybe on the 10 bits format. > > You can also checkout YUView (https://github.com/IENT/YUView) to get > information on media files structure, but I have had issues with HEVC support > lately. oh great. will do and report!
Update to my last email: > Wiadomość napisana przez Piotr Oniszczuk <piotr.oniszczuk@gmail.com> w dniu 30 sty 2025, o godz. 11:46: > > > I’m using: > https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux/-/commit/bc47c445bfd9586115e9bcf5f231c5a5c5f0f828 > https://gitlab.collabora.com/hardware-enablement/rockchip-3588/linux/-/commit/9a9fd791513bc0d02c2242c88f23b41bd47de30a > > I see there is: https://gitlab.collabora.com/detlev/linux/-/commits/add-rkvdec2-driver-iommu > Let me update iommu code to above branch and report back here how it goes in 3588 and 356x I updated to https://gitlab.collabora.com/detlev/linux/-/commits/add-rkvdec2-driver-iommu and basically i see no change (the same iommu errors on 356x; 3588 all clean) > > >> >> You can also checkout YUView (https://github.com/IENT/YUView) to get >> information on media files structure, but I have had issues with HEVC support >> lately. > > oh great. will do and report! > Huh - here in need to switch to Linux as on maxOS i have issue with ffmpeg ver. mismatch in YUView Will report next days….