| Message ID | 20250506-qcom-apcs-mailbox-cc-v1-1-b54dddb150a5@linaro.org |
|---|---|
| State | New |
| Headers | show |
| Series | mailbox: qcom-apcs-ipc: Avoid circular dependency with clock controller | expand |
On Tue, May 06, 2025 at 03:10:08PM +0200, Stephan Gerhold wrote: > APCS "global" is sort of a "miscellaneous" hardware block that combines > multiple registers inside the application processor subsystem. Two distinct > use cases are currently stuffed together in a single device tree node: > > - Mailbox: to communicate with other remoteprocs in the system. > - Clock: for controlling the CPU frequency. > > These two use cases have unavoidable circular dependencies: the mailbox is > needed as early as possible during boot to start controlling shared > resources like clocks and power domains, while the clock controller needs > one of these shared clocks as its parent. Currently, there is no way to > distinguish these two use cases for generic mechanisms like fw_devlink. > > This is currently blocking conversion of the deprecated custom "qcom,ipc" > properties to the standard "mboxes", see e.g. commit d92e9ea2f0f9 > ("arm64: dts: qcom: msm8939: revert use of APCS mbox for RPM"): > 1. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>; > 2. The clock controller inside &apcs1_mbox needs > clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>. > 3. &rpmcc is a child of remoteproc &rpm > > The mailbox itself does not need any clocks and should probe early to > unblock the rest of the boot process. The "clocks" are only needed for the > separate clock controller. In Linux, these are already two separate drivers > that can probe independently. > Why does this circular dependency need to be broken in the DeviceTree representation? As you describe, the mailbox probes and register the mailbox controller and it registers the clock controller. The mailbox device isn't affected by the clock controller failing to find rpmcc... Regards, Bjorn > Break up the circular dependency chain in the device tree by separating the > clock controller into a separate child node. Deprecate the old approach of > specifying the clock properties as part of the root node, but keep them for > backwards compatibility. > > Signed-off-by: Stephan Gerhold <stephan.gerhold@linaro.org> > --- > .../bindings/mailbox/qcom,apcs-kpss-global.yaml | 169 ++++++++++++++------- > 1 file changed, 118 insertions(+), 51 deletions(-) > > diff --git a/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml b/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml > index a58a018f3f7b9f8edd70d7c1bd137844ff2549df..3a0a304bb65a68b2d4a1df79b3243ddac6bf88b2 100644 > --- a/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml > +++ b/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml > @@ -72,6 +72,7 @@ properties: > description: phandles to the parent clocks of the clock driver > minItems: 2 > maxItems: 3 > + deprecated: true > > '#mbox-cells': > const: 1 > @@ -82,6 +83,23 @@ properties: > clock-names: > minItems: 2 > maxItems: 3 > + deprecated: true > + > + clock-controller: > + type: object > + additionalProperties: false > + properties: > + clocks: > + description: phandles to the parent clocks of the clock driver > + minItems: 2 > + maxItems: 3 > + > + '#clock-cells': > + enum: [0, 1] > + > + clock-names: > + minItems: 2 > + maxItems: 3 > > required: > - compatible > @@ -90,6 +108,76 @@ required: > > additionalProperties: false > > +# Clocks should be specified either on the parent node or on the child node > +oneOf: > + - required: > + - clock-controller > + properties: > + clocks: false > + clock-names: false > + '#clock-cells': false > + - properties: > + clock-controller: false > + > +$defs: > + msm8916-apcs-clock-controller: > + properties: > + clocks: > + items: > + - description: primary pll parent of the clock driver > + - description: auxiliary parent > + clock-names: > + items: > + - const: pll > + - const: aux > + '#clock-cells': > + const: 0 > + > + msm8939-apcs-clock-controller: > + properties: > + clocks: > + items: > + - description: primary pll parent of the clock driver > + - description: auxiliary parent > + - description: reference clock > + clock-names: > + items: > + - const: pll > + - const: aux > + - const: ref > + '#clock-cells': > + const: 0 > + > + sdx55-apcs-clock-controller: > + properties: > + clocks: > + items: > + - description: reference clock > + - description: primary pll parent of the clock driver > + - description: auxiliary parent > + clock-names: > + items: > + - const: ref > + - const: pll > + - const: aux > + '#clock-cells': > + const: 0 > + > + ipq6018-apcs-clock-controller: > + properties: > + clocks: > + items: > + - description: primary pll parent of the clock driver > + - description: XO clock > + - description: GCC GPLL0 clock source > + clock-names: > + items: > + - const: pll > + - const: xo > + - const: gpll0 > + '#clock-cells': > + const: 1 > + > allOf: > - if: > properties: > @@ -98,15 +186,10 @@ allOf: > enum: > - qcom,msm8916-apcs-kpss-global > then: > + $ref: "#/$defs/msm8916-apcs-clock-controller" > properties: > - clocks: > - items: > - - description: primary pll parent of the clock driver > - - description: auxiliary parent > - clock-names: > - items: > - - const: pll > - - const: aux > + clock-controller: > + $ref: "#/$defs/msm8916-apcs-clock-controller" > > - if: > properties: > @@ -115,17 +198,10 @@ allOf: > enum: > - qcom,msm8939-apcs-kpss-global > then: > + $ref: "#/$defs/msm8939-apcs-clock-controller" > properties: > - clocks: > - items: > - - description: primary pll parent of the clock driver > - - description: auxiliary parent > - - description: reference clock > - clock-names: > - items: > - - const: pll > - - const: aux > - - const: ref > + clock-controller: > + $ref: "#/$defs/msm8939-apcs-clock-controller" > > - if: > properties: > @@ -134,17 +210,10 @@ allOf: > enum: > - qcom,sdx55-apcs-gcc > then: > + $ref: "#/$defs/sdx55-apcs-clock-controller" > properties: > - clocks: > - items: > - - description: reference clock > - - description: primary pll parent of the clock driver > - - description: auxiliary parent > - clock-names: > - items: > - - const: ref > - - const: pll > - - const: aux > + clock-controller: > + $ref: "#/$defs/sdx55-apcs-clock-controller" > > - if: > properties: > @@ -153,17 +222,10 @@ allOf: > enum: > - qcom,ipq6018-apcs-apps-global > then: > + $ref: "#/$defs/ipq6018-apcs-clock-controller" > properties: > - clocks: > - items: > - - description: primary pll parent of the clock driver > - - description: XO clock > - - description: GCC GPLL0 clock source > - clock-names: > - items: > - - const: pll > - - const: xo > - - const: gpll0 > + clock-controller: > + $ref: "#/$defs/ipq6018-apcs-clock-controller" > > - if: > properties: > @@ -179,19 +241,7 @@ allOf: > properties: > clocks: false > clock-names: false > - > - - if: > - properties: > - compatible: > - contains: > - enum: > - - qcom,ipq6018-apcs-apps-global > - then: > - properties: > - '#clock-cells': > - const: 1 > - else: > - properties: > + clock-controller: false > '#clock-cells': > const: 0 > > @@ -216,6 +266,23 @@ examples: > }; > > # Example apcs with qcs404 > + - | > + #define GCC_APSS_AHB_CLK_SRC 1 > + #define GCC_GPLL0_AO_OUT_MAIN 123 > + mailbox@b011000 { > + compatible = "qcom,qcs404-apcs-apps-global", > + "qcom,msm8916-apcs-kpss-global", "syscon"; > + reg = <0x0b011000 0x1000>; > + #mbox-cells = <1>; > + > + apcs_clk: clock-controller { > + clocks = <&apcs_hfpll>, <&gcc GCC_GPLL0_AO_OUT_MAIN>; > + clock-names = "pll", "aux"; > + #clock-cells = <0>; > + }; > + }; > + > + # Example apcs with qcs404 (deprecated: use clock-controller subnode) > - | > #define GCC_APSS_AHB_CLK_SRC 1 > #define GCC_GPLL0_AO_OUT_MAIN 123 > > -- > 2.47.2 >
On Sun, May 11, 2025 at 05:48:11PM -0500, Bjorn Andersson wrote: > On Tue, May 06, 2025 at 03:10:08PM +0200, Stephan Gerhold wrote: > > APCS "global" is sort of a "miscellaneous" hardware block that combines > > multiple registers inside the application processor subsystem. Two distinct > > use cases are currently stuffed together in a single device tree node: > > > > - Mailbox: to communicate with other remoteprocs in the system. > > - Clock: for controlling the CPU frequency. > > > > These two use cases have unavoidable circular dependencies: the mailbox is > > needed as early as possible during boot to start controlling shared > > resources like clocks and power domains, while the clock controller needs > > one of these shared clocks as its parent. Currently, there is no way to > > distinguish these two use cases for generic mechanisms like fw_devlink. > > > > This is currently blocking conversion of the deprecated custom "qcom,ipc" > > properties to the standard "mboxes", see e.g. commit d92e9ea2f0f9 > > ("arm64: dts: qcom: msm8939: revert use of APCS mbox for RPM"): > > 1. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>; > > 2. The clock controller inside &apcs1_mbox needs > > clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>. > > 3. &rpmcc is a child of remoteproc &rpm > > > > The mailbox itself does not need any clocks and should probe early to > > unblock the rest of the boot process. The "clocks" are only needed for the > > separate clock controller. In Linux, these are already two separate drivers > > that can probe independently. > > > > Why does this circular dependency need to be broken in the DeviceTree > representation? > > As you describe, the mailbox probes and register the mailbox controller > and it registers the clock controller. The mailbox device isn't affected > by the clock controller failing to find rpmcc... > That's right, but the problem is that the probe() function of the mailbox driver won't be called at all. The device tree *looks* like the mailbox depends on the clock, so fw_devlink tries to defer probing until the clock is probed (which won't ever happen, because the mailbox is needed to make the clock available). I'm not sure why fw_devlink doesn't detect this cycle and tries to probe them anyway, but fact is that we need to split this up in order to avoid warnings and have the supplies/consumers set up properly. Those device links are created based on the device tree and not the drivers. Thanks, Stephan
On Tue, May 13, 2025 at 02:16:59PM +0100, Stephan Gerhold wrote: > On Sun, May 11, 2025 at 05:48:11PM -0500, Bjorn Andersson wrote: > > On Tue, May 06, 2025 at 03:10:08PM +0200, Stephan Gerhold wrote: > > > APCS "global" is sort of a "miscellaneous" hardware block that combines > > > multiple registers inside the application processor subsystem. Two distinct > > > use cases are currently stuffed together in a single device tree node: > > > > > > - Mailbox: to communicate with other remoteprocs in the system. > > > - Clock: for controlling the CPU frequency. > > > > > > These two use cases have unavoidable circular dependencies: the mailbox is > > > needed as early as possible during boot to start controlling shared > > > resources like clocks and power domains, while the clock controller needs > > > one of these shared clocks as its parent. Currently, there is no way to > > > distinguish these two use cases for generic mechanisms like fw_devlink. > > > > > > This is currently blocking conversion of the deprecated custom "qcom,ipc" > > > properties to the standard "mboxes", see e.g. commit d92e9ea2f0f9 > > > ("arm64: dts: qcom: msm8939: revert use of APCS mbox for RPM"): > > > 1. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>; > > > 2. The clock controller inside &apcs1_mbox needs > > > clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>. > > > 3. &rpmcc is a child of remoteproc &rpm > > > > > > The mailbox itself does not need any clocks and should probe early to > > > unblock the rest of the boot process. The "clocks" are only needed for the > > > separate clock controller. In Linux, these are already two separate drivers > > > that can probe independently. > > > > > > > Why does this circular dependency need to be broken in the DeviceTree > > representation? > > > > As you describe, the mailbox probes and register the mailbox controller > > and it registers the clock controller. The mailbox device isn't affected > > by the clock controller failing to find rpmcc... > > > > That's right, but the problem is that the probe() function of the > mailbox driver won't be called at all. The device tree *looks* like the > mailbox depends on the clock, so fw_devlink tries to defer probing until > the clock is probed (which won't ever happen, because the mailbox is > needed to make the clock available). > > I'm not sure why fw_devlink doesn't detect this cycle and tries to probe > them anyway, but fact is that we need to split this up in order to avoid > warnings and have the supplies/consumers set up properly. Those device > links are created based on the device tree and not the drivers. Does "post-init-providers" providers solve your problem? Rob
On Wed, May 14, 2025 at 11:08:41AM -0500, Rob Herring wrote: > On Tue, May 13, 2025 at 02:16:59PM +0100, Stephan Gerhold wrote: > > On Sun, May 11, 2025 at 05:48:11PM -0500, Bjorn Andersson wrote: > > > On Tue, May 06, 2025 at 03:10:08PM +0200, Stephan Gerhold wrote: > > > > APCS "global" is sort of a "miscellaneous" hardware block that combines > > > > multiple registers inside the application processor subsystem. Two distinct > > > > use cases are currently stuffed together in a single device tree node: > > > > > > > > - Mailbox: to communicate with other remoteprocs in the system. > > > > - Clock: for controlling the CPU frequency. > > > > > > > > These two use cases have unavoidable circular dependencies: the mailbox is > > > > needed as early as possible during boot to start controlling shared > > > > resources like clocks and power domains, while the clock controller needs > > > > one of these shared clocks as its parent. Currently, there is no way to > > > > distinguish these two use cases for generic mechanisms like fw_devlink. > > > > > > > > This is currently blocking conversion of the deprecated custom "qcom,ipc" > > > > properties to the standard "mboxes", see e.g. commit d92e9ea2f0f9 > > > > ("arm64: dts: qcom: msm8939: revert use of APCS mbox for RPM"): > > > > 1. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>; > > > > 2. The clock controller inside &apcs1_mbox needs > > > > clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>. > > > > 3. &rpmcc is a child of remoteproc &rpm > > > > > > > > The mailbox itself does not need any clocks and should probe early to > > > > unblock the rest of the boot process. The "clocks" are only needed for the > > > > separate clock controller. In Linux, these are already two separate drivers > > > > that can probe independently. > > > > > > > > > > Why does this circular dependency need to be broken in the DeviceTree > > > representation? > > > > > > As you describe, the mailbox probes and register the mailbox controller > > > and it registers the clock controller. The mailbox device isn't affected > > > by the clock controller failing to find rpmcc... > > > > > > > That's right, but the problem is that the probe() function of the > > mailbox driver won't be called at all. The device tree *looks* like the > > mailbox depends on the clock, so fw_devlink tries to defer probing until > > the clock is probed (which won't ever happen, because the mailbox is > > needed to make the clock available). > > > > I'm not sure why fw_devlink doesn't detect this cycle and tries to probe > > them anyway, but fact is that we need to split this up in order to avoid > > warnings and have the supplies/consumers set up properly. Those device > > links are created based on the device tree and not the drivers. > > Does "post-init-providers" providers solve your problem? > I would expect that it does, but it feels like the wrong solution to the problem to me. The clock is not really a post-init provider: It's not consumed at all by the mailbox and needed immediately to initialize the clock controller. The real problem in my opinion is that we're describing two essentially distinct devices/drivers in a single device node, and there is no way to distinguish that. By splitting up the two distinct components into separate device tree nodes, the relation between the providers/consumers is clearly described. Thanks, Stephan
+Saravana On Wed, May 21, 2025 at 11:20:40AM +0200, Krzysztof Kozlowski wrote: > On Wed, May 14, 2025 at 10:12:44PM GMT, Stephan Gerhold wrote: > > > > > > The mailbox itself does not need any clocks and should probe early to > > ... so probe it early. > > > > > > > unblock the rest of the boot process. The "clocks" are only needed for the > > > > > > separate clock controller. In Linux, these are already two separate drivers > > > > > > that can probe independently. > > They can probe later, no problem and DT does not stop that. Linux, not > DT, controls the ways of probing of devices and their children. > > > > > > > > > > > > > > > > > Why does this circular dependency need to be broken in the DeviceTree > > > > > representation? > > > > > > > > > > As you describe, the mailbox probes and register the mailbox controller > > > > > and it registers the clock controller. The mailbox device isn't affected > > > > > by the clock controller failing to find rpmcc... > > > > > > > > > > > > > That's right, but the problem is that the probe() function of the > > > > mailbox driver won't be called at all. The device tree *looks* like the > > > > mailbox depends on the clock, so fw_devlink tries to defer probing until > > > > the clock is probed (which won't ever happen, because the mailbox is > > > > needed to make the clock available). > > > > > > > > I'm not sure why fw_devlink doesn't detect this cycle and tries to probe > > > > them anyway, but fact is that we need to split this up in order to avoid > > > > warnings and have the supplies/consumers set up properly. Those device > > > > links are created based on the device tree and not the drivers. > > > > > > Does "post-init-providers" providers solve your problem? > > > > > > > I would expect that it does, but it feels like the wrong solution to the > > problem to me. The clock is not really a post-init provider: It's not > > consumed at all by the mailbox and needed immediately to initialize the > > clock controller. The real problem in my opinion is that we're > > describing two essentially distinct devices/drivers in a single device > > node, and there is no way to distinguish that. > > > > By splitting up the two distinct components into separate device tree > > nodes, the relation between the providers/consumers is clearly > > described. > > You can split devices without splitting the nodes. I do not see reason > why the DT is the problem here. > The Linux drivers for this particular mailbox/clock controller already work exactly the way you propose. They are split into two devices that can probe independently. The problem is outside of the drivers, because fw_devlink in Linux blocks probing until all resources specified in the device tree nodes become available. fw_devlink has no knowledge that the mailbox described by this peculiar device tree node does not actually need the clocks: apcs1_mbox: mailbox@b011000 { compatible = "qcom,msm8939-apcs-kpss-global", "syscon"; reg = <0x0b011000 0x1000>; #mbox-cells = <1>; clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>; clock-names = "pll", "aux", "ref"; #clock-cells = <0>; }; Without device-specific quirks in fw_devlink, the fact that these clocks are only used by an unrelated clock controller only becomes clear if we split the device tree node like I propose in this series: apcs1_mbox: mailbox@b011000 { compatible = "qcom,msm8939-apcs-kpss-global", "syscon"; reg = <0x0b011000 0x1000>; #mbox-cells = <1>; apcs1_clk: clock-controller { clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>; clock-names = "pll", "aux", "ref"; #clock-cells = <0>; }; }; It is easy to say that the problem is in Linux (and not the DT), but unless you are suggesting to remove fw_devlink from Linux, or to add more device-specific quirks to the generic fw_devlink code, I'm only aware of the following two options to make this work (both already discussed in this email thread): 1. post-init-providers (as suggested by Rob): post-init-providers = <&a53pll_c1>, <&gcc>, <&rpmcc>; To repeat my previous email: IMHO this is a crude workaround for this situation. The clock is not really a post-init provider: It's not consumed at all by the mailbox and needed immediately to initialize the clock controller. With this approach, there are no device links created for the clocks, so we don't get the proper probe/suspend ordering that fw_devlink normally provides. 2. Split up device tree node (this patch series): With this approach, the mailbox can probe early and the clock controller child device gets the expected consumer/supplier device links to the clocks. IMHO this is the cleanest solution to go for. @Saravana: Is there any other option that I missed? Or perhaps you have any other suggestions how we should handle this? To summarize the series and previous emails, the dependency cycle that was in msm8939.dtsi before commit d92e9ea2f0f9 ("arm64: dts: qcom: msm8939: revert use of APCS mbox for RPM") is: 1. The clock controller inside &apcs1_mbox needs clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>. 2. &rpmcc is a child of remoteproc &rpm 3. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>; This is not a real dependency cycle, the clocks in the mailbox@ node are not needed for the mailbox. They are only used and needed for the clock controller child device that makes use of the same device tree node. At runtime this cycle currently results in none of the devices probing: [ 13.281637] platform remoteproc: deferred probe pending: qcom-rpm-proc: Failed to register smd-edge [ 13.296257] platform b011000.mailbox: deferred probe pending: platform: supplier b016000.clock not ready [ 13.308397] platform b016000.clock: deferred probe pending: platform: wait for supplier /remoteproc/smd-edge/rpm-requests/clock-controller Thanks, Stephan
On Fri, May 23, 2025 at 10:42:01AM +0200, Krzysztof Kozlowski wrote: > On Wed, May 14, 2025 at 10:12:44PM GMT, Stephan Gerhold wrote: > > On Wed, May 14, 2025 at 11:08:41AM -0500, Rob Herring wrote: > > > On Tue, May 13, 2025 at 02:16:59PM +0100, Stephan Gerhold wrote: > > > > On Sun, May 11, 2025 at 05:48:11PM -0500, Bjorn Andersson wrote: > > > > > On Tue, May 06, 2025 at 03:10:08PM +0200, Stephan Gerhold wrote: > > > > > > APCS "global" is sort of a "miscellaneous" hardware block that combines > > > > > > multiple registers inside the application processor subsystem. Two distinct > > > > > > use cases are currently stuffed together in a single device tree node: > > > > > > > > > > > > - Mailbox: to communicate with other remoteprocs in the system. > > > > > > - Clock: for controlling the CPU frequency. > > > > > > > > > > > > These two use cases have unavoidable circular dependencies: the mailbox is > > > > > > needed as early as possible during boot to start controlling shared > > > > > > resources like clocks and power domains, while the clock controller needs > > > > > > one of these shared clocks as its parent. Currently, there is no way to > > > > > > distinguish these two use cases for generic mechanisms like fw_devlink. > > > > > > > > > > > > This is currently blocking conversion of the deprecated custom "qcom,ipc" > > > > > > properties to the standard "mboxes", see e.g. commit d92e9ea2f0f9 > > > > > > ("arm64: dts: qcom: msm8939: revert use of APCS mbox for RPM"): > > > > > > 1. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>; > > > > > > 2. The clock controller inside &apcs1_mbox needs > > > > > > clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>. > > > > > > 3. &rpmcc is a child of remoteproc &rpm > > > > > > > > > > > > The mailbox itself does not need any clocks and should probe early to > > > > > > unblock the rest of the boot process. The "clocks" are only needed for the > > > > > > separate clock controller. In Linux, these are already two separate drivers > > > > > > that can probe independently. > > > > > > > > > > > > > > > > Why does this circular dependency need to be broken in the DeviceTree > > > > > representation? > > > > > > > > > > As you describe, the mailbox probes and register the mailbox controller > > > > > and it registers the clock controller. The mailbox device isn't affected > > > > > by the clock controller failing to find rpmcc... > > > > > > > > > > > > > That's right, but the problem is that the probe() function of the > > > > mailbox driver won't be called at all. The device tree *looks* like the > > > > mailbox depends on the clock, so fw_devlink tries to defer probing until > > > > the clock is probed (which won't ever happen, because the mailbox is > > > > needed to make the clock available). > > > > > > > > I'm not sure why fw_devlink doesn't detect this cycle and tries to probe > > > > them anyway, but fact is that we need to split this up in order to avoid > > > > warnings and have the supplies/consumers set up properly. Those device > > > > links are created based on the device tree and not the drivers. > > > > > > Does "post-init-providers" providers solve your problem? > > > > > > > I would expect that it does, but it feels like the wrong solution to the > > problem to me. The clock is not really a post-init provider: It's not > > But the entire node (so the mbox containing clocks) is a provider. This > looks exactly like the case for post-init or do I miss here something. > First, I do not see circular dependencies in the DT. > Please see my reply from yesterday, I have explained the disadvantages of using post-init-provider there and also showed the problematic circular dependency again [1]. Thanks, Stephan [1]: https://lore.kernel.org/linux-arm-msm/aC-AqDa8cjq2AYeM@linaro.org/
On Tue, May 06, 2025 at 03:10:08PM GMT, Stephan Gerhold wrote: > APCS "global" is sort of a "miscellaneous" hardware block that combines > multiple registers inside the application processor subsystem. Two distinct > use cases are currently stuffed together in a single device tree node: > > - Mailbox: to communicate with other remoteprocs in the system. > - Clock: for controlling the CPU frequency. Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@linaro.org> Best regards, Krzysztof
On Fri, May 23, 2025 at 11:06:04AM +0200, Krzysztof Kozlowski wrote: > On Thu, May 22, 2025 at 09:53:12PM GMT, Stephan Gerhold wrote: > > +Saravana > > > > On Wed, May 21, 2025 at 11:20:40AM +0200, Krzysztof Kozlowski wrote: > > > On Wed, May 14, 2025 at 10:12:44PM GMT, Stephan Gerhold wrote: > > > > > > > > The mailbox itself does not need any clocks and should probe early to > > > > > > ... so probe it early. > > > > > > > > > > > unblock the rest of the boot process. The "clocks" are only needed for the > > > > > > > > separate clock controller. In Linux, these are already two separate drivers > > > > > > > > that can probe independently. > > > > > > They can probe later, no problem and DT does not stop that. Linux, not > > > DT, controls the ways of probing of devices and their children. > > > > > > > > > > > > > > > > > > > > > > > > > Why does this circular dependency need to be broken in the DeviceTree > > > > > > > representation? > > > > > > > > > > > > > > As you describe, the mailbox probes and register the mailbox controller > > > > > > > and it registers the clock controller. The mailbox device isn't affected > > > > > > > by the clock controller failing to find rpmcc... > > > > > > > > > > > > > > > > > > > That's right, but the problem is that the probe() function of the > > > > > > mailbox driver won't be called at all. The device tree *looks* like the > > > > > > mailbox depends on the clock, so fw_devlink tries to defer probing until > > > > > > the clock is probed (which won't ever happen, because the mailbox is > > > > > > needed to make the clock available). > > > > > > > > > > > > I'm not sure why fw_devlink doesn't detect this cycle and tries to probe > > > > > > them anyway, but fact is that we need to split this up in order to avoid > > > > > > warnings and have the supplies/consumers set up properly. Those device > > > > > > links are created based on the device tree and not the drivers. > > > > > > > > > > Does "post-init-providers" providers solve your problem? > > > > > > > > > > > > > I would expect that it does, but it feels like the wrong solution to the > > > > problem to me. The clock is not really a post-init provider: It's not > > > > consumed at all by the mailbox and needed immediately to initialize the > > > > clock controller. The real problem in my opinion is that we're > > > > describing two essentially distinct devices/drivers in a single device > > > > node, and there is no way to distinguish that. > > > > > > > > By splitting up the two distinct components into separate device tree > > > > nodes, the relation between the providers/consumers is clearly > > > > described. > > > > > > You can split devices without splitting the nodes. I do not see reason > > > why the DT is the problem here. > > > > > > > The Linux drivers for this particular mailbox/clock controller already > > work exactly the way you propose. They are split into two devices that > > can probe independently. > > > > The problem is outside of the drivers, because fw_devlink in Linux > > blocks probing until all resources specified in the device tree nodes > > become available. fw_devlink has no knowledge that the mailbox described > > by this peculiar device tree node does not actually need the clocks: > > > > apcs1_mbox: mailbox@b011000 { > > compatible = "qcom,msm8939-apcs-kpss-global", "syscon"; > > reg = <0x0b011000 0x1000>; > > #mbox-cells = <1>; > > clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>; > > clock-names = "pll", "aux", "ref"; > > #clock-cells = <0>; > > }; > > > > Without device-specific quirks in fw_devlink, the fact that these clocks > > are only used by an unrelated clock controller only becomes clear if we > > split the device tree node like I propose in this series: > > > > apcs1_mbox: mailbox@b011000 { > > compatible = "qcom,msm8939-apcs-kpss-global", "syscon"; > > reg = <0x0b011000 0x1000>; > > #mbox-cells = <1>; > > > > apcs1_clk: clock-controller { > > clocks = <&a53pll_c1>, <&gcc GPLL0_VOTE>, <&rpmcc RPM_SMD_XO_CLK_SRC>; > > clock-names = "pll", "aux", "ref"; > > #clock-cells = <0>; > > }; > > }; > > Above code suggests that clocks are not needed for the mailbox at all. > You need to be really sure of that. If that's the case, then this > description looks like correct hardware description, more detailed then > the first case, though. > Yes, these clocks are not needed for the mailbox. > > > > It is easy to say that the problem is in Linux (and not the DT), but > > unless you are suggesting to remove fw_devlink from Linux, or to add > > more device-specific quirks to the generic fw_devlink code, I'm only > > aware of the following two options to make this work (both already > > discussed in this email thread): > > > > 1. post-init-providers (as suggested by Rob): > > > > post-init-providers = <&a53pll_c1>, <&gcc>, <&rpmcc>; > > > > To repeat my previous email: IMHO this is a crude workaround for > > this situation. The clock is not really a post-init provider: It's > > not consumed at all by the mailbox and needed immediately to > > initialize the clock controller. > > Clock is a provider - it has clock-cells - and it is post-init, because > mailbox (parent) does not need it to initialize itself. Only part of its > functionality - clocks - need it. > Right. > > > > With this approach, there are no device links created for the > > clocks, so we don't get the proper probe/suspend ordering that > > fw_devlink normally provides. > > This probably could be fixed in the Linux? > This is probably something for Saravana to answer, but I think post-init-providers is more intended to break up actual circular dependencies, where you want to omit the device links for one of the sides. For this specific case this is not necessary, because we can avoid creating the cycle by describing the hardware more accurately. > > > > 2. Split up device tree node (this patch series): With this approach, > > the mailbox can probe early and the clock controller child device > > gets the expected consumer/supplier device links to the clocks. IMHO > > this is the cleanest solution to go for. > > > > @Saravana: Is there any other option that I missed? Or perhaps you have > > any other suggestions how we should handle this? > > > > To summarize the series and previous emails, the dependency cycle that > > was in msm8939.dtsi before commit d92e9ea2f0f9 ("arm64: dts: qcom: > > msm8939: revert use of APCS mbox for RPM") is: > > > > 1. The clock controller inside &apcs1_mbox needs > > clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>. > > 2. &rpmcc is a child of remoteproc &rpm > > 3. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>; > > > > This is not a real dependency cycle, the clocks in the mailbox@ node are > > not needed for the mailbox. They are only used and needed for the clock > > controller child device that makes use of the same device tree node. > > > > At runtime this cycle currently results in none of the devices probing: > > > > [ 13.281637] platform remoteproc: deferred probe pending: qcom-rpm-proc: Failed to register smd-edge > > [ 13.296257] platform b011000.mailbox: deferred probe pending: platform: supplier b016000.clock not ready > > [ 13.308397] platform b016000.clock: deferred probe pending: platform: wait for supplier /remoteproc/smd-edge/rpm-requests/clock-controller > > > > Except missing dev_links you mentioned, this feels for me like job for > post-init-providers (option 1), but I am also fine with option 2, > because it fees like more appropriate hardware description. > Great, thanks for the review! Stephan
diff --git a/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml b/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml index a58a018f3f7b9f8edd70d7c1bd137844ff2549df..3a0a304bb65a68b2d4a1df79b3243ddac6bf88b2 100644 --- a/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml +++ b/Documentation/devicetree/bindings/mailbox/qcom,apcs-kpss-global.yaml @@ -72,6 +72,7 @@ properties: description: phandles to the parent clocks of the clock driver minItems: 2 maxItems: 3 + deprecated: true '#mbox-cells': const: 1 @@ -82,6 +83,23 @@ properties: clock-names: minItems: 2 maxItems: 3 + deprecated: true + + clock-controller: + type: object + additionalProperties: false + properties: + clocks: + description: phandles to the parent clocks of the clock driver + minItems: 2 + maxItems: 3 + + '#clock-cells': + enum: [0, 1] + + clock-names: + minItems: 2 + maxItems: 3 required: - compatible @@ -90,6 +108,76 @@ required: additionalProperties: false +# Clocks should be specified either on the parent node or on the child node +oneOf: + - required: + - clock-controller + properties: + clocks: false + clock-names: false + '#clock-cells': false + - properties: + clock-controller: false + +$defs: + msm8916-apcs-clock-controller: + properties: + clocks: + items: + - description: primary pll parent of the clock driver + - description: auxiliary parent + clock-names: + items: + - const: pll + - const: aux + '#clock-cells': + const: 0 + + msm8939-apcs-clock-controller: + properties: + clocks: + items: + - description: primary pll parent of the clock driver + - description: auxiliary parent + - description: reference clock + clock-names: + items: + - const: pll + - const: aux + - const: ref + '#clock-cells': + const: 0 + + sdx55-apcs-clock-controller: + properties: + clocks: + items: + - description: reference clock + - description: primary pll parent of the clock driver + - description: auxiliary parent + clock-names: + items: + - const: ref + - const: pll + - const: aux + '#clock-cells': + const: 0 + + ipq6018-apcs-clock-controller: + properties: + clocks: + items: + - description: primary pll parent of the clock driver + - description: XO clock + - description: GCC GPLL0 clock source + clock-names: + items: + - const: pll + - const: xo + - const: gpll0 + '#clock-cells': + const: 1 + allOf: - if: properties: @@ -98,15 +186,10 @@ allOf: enum: - qcom,msm8916-apcs-kpss-global then: + $ref: "#/$defs/msm8916-apcs-clock-controller" properties: - clocks: - items: - - description: primary pll parent of the clock driver - - description: auxiliary parent - clock-names: - items: - - const: pll - - const: aux + clock-controller: + $ref: "#/$defs/msm8916-apcs-clock-controller" - if: properties: @@ -115,17 +198,10 @@ allOf: enum: - qcom,msm8939-apcs-kpss-global then: + $ref: "#/$defs/msm8939-apcs-clock-controller" properties: - clocks: - items: - - description: primary pll parent of the clock driver - - description: auxiliary parent - - description: reference clock - clock-names: - items: - - const: pll - - const: aux - - const: ref + clock-controller: + $ref: "#/$defs/msm8939-apcs-clock-controller" - if: properties: @@ -134,17 +210,10 @@ allOf: enum: - qcom,sdx55-apcs-gcc then: + $ref: "#/$defs/sdx55-apcs-clock-controller" properties: - clocks: - items: - - description: reference clock - - description: primary pll parent of the clock driver - - description: auxiliary parent - clock-names: - items: - - const: ref - - const: pll - - const: aux + clock-controller: + $ref: "#/$defs/sdx55-apcs-clock-controller" - if: properties: @@ -153,17 +222,10 @@ allOf: enum: - qcom,ipq6018-apcs-apps-global then: + $ref: "#/$defs/ipq6018-apcs-clock-controller" properties: - clocks: - items: - - description: primary pll parent of the clock driver - - description: XO clock - - description: GCC GPLL0 clock source - clock-names: - items: - - const: pll - - const: xo - - const: gpll0 + clock-controller: + $ref: "#/$defs/ipq6018-apcs-clock-controller" - if: properties: @@ -179,19 +241,7 @@ allOf: properties: clocks: false clock-names: false - - - if: - properties: - compatible: - contains: - enum: - - qcom,ipq6018-apcs-apps-global - then: - properties: - '#clock-cells': - const: 1 - else: - properties: + clock-controller: false '#clock-cells': const: 0 @@ -216,6 +266,23 @@ examples: }; # Example apcs with qcs404 + - | + #define GCC_APSS_AHB_CLK_SRC 1 + #define GCC_GPLL0_AO_OUT_MAIN 123 + mailbox@b011000 { + compatible = "qcom,qcs404-apcs-apps-global", + "qcom,msm8916-apcs-kpss-global", "syscon"; + reg = <0x0b011000 0x1000>; + #mbox-cells = <1>; + + apcs_clk: clock-controller { + clocks = <&apcs_hfpll>, <&gcc GCC_GPLL0_AO_OUT_MAIN>; + clock-names = "pll", "aux"; + #clock-cells = <0>; + }; + }; + + # Example apcs with qcs404 (deprecated: use clock-controller subnode) - | #define GCC_APSS_AHB_CLK_SRC 1 #define GCC_GPLL0_AO_OUT_MAIN 123
APCS "global" is sort of a "miscellaneous" hardware block that combines multiple registers inside the application processor subsystem. Two distinct use cases are currently stuffed together in a single device tree node: - Mailbox: to communicate with other remoteprocs in the system. - Clock: for controlling the CPU frequency. These two use cases have unavoidable circular dependencies: the mailbox is needed as early as possible during boot to start controlling shared resources like clocks and power domains, while the clock controller needs one of these shared clocks as its parent. Currently, there is no way to distinguish these two use cases for generic mechanisms like fw_devlink. This is currently blocking conversion of the deprecated custom "qcom,ipc" properties to the standard "mboxes", see e.g. commit d92e9ea2f0f9 ("arm64: dts: qcom: msm8939: revert use of APCS mbox for RPM"): 1. remoteproc &rpm needs mboxes = <&apcs1_mbox 8>; 2. The clock controller inside &apcs1_mbox needs clocks = <&rpmcc RPM_SMD_XO_CLK_SRC>. 3. &rpmcc is a child of remoteproc &rpm The mailbox itself does not need any clocks and should probe early to unblock the rest of the boot process. The "clocks" are only needed for the separate clock controller. In Linux, these are already two separate drivers that can probe independently. Break up the circular dependency chain in the device tree by separating the clock controller into a separate child node. Deprecate the old approach of specifying the clock properties as part of the root node, but keep them for backwards compatibility. Signed-off-by: Stephan Gerhold <stephan.gerhold@linaro.org> --- .../bindings/mailbox/qcom,apcs-kpss-global.yaml | 169 ++++++++++++++------- 1 file changed, 118 insertions(+), 51 deletions(-)