diff mbox series

[1/2] rcutorture: Don't disable softirqs with preemption disabled when PREEMPT_RT

Message ID 20210803225437.3612591-2-valentin.schneider@arm.com
State Superseded
Headers show
Series [1/2] rcutorture: Don't disable softirqs with preemption disabled when PREEMPT_RT | expand

Commit Message

Valentin Schneider Aug. 3, 2021, 10:54 p.m. UTC 
Running RCU torture with CONFIG_PREEMPT_RT under v5.13-rt1 triggers:

[   10.821700] DEBUG_LOCKS_WARN_ON(this_cpu_read(softirq_ctrl.cnt))
[   10.821716] WARNING: CPU: 5 PID: 137 at kernel/softirq.c:173 __local_bh_disable_ip (kernel/softirq.c:173 (discriminator 31))
[   10.821739] Modules linked in:
[   10.821749] CPU: 5 PID: 137 Comm: rcu_torture_rea Not tainted 5.13.0-rt1-00005-g08bbda29766a #129
[   10.821759] Hardware name: ARM Juno development board (r0) (DT)
[   10.821765] pstate: 60000005 (nZCv daif -PAN -UAO -TCO BTYPE=--)
[   10.821938] Call trace:
[   10.821941] __local_bh_disable_ip (kernel/softirq.c:173 (discriminator 31))
[   10.821950] rcutorture_one_extend (./include/linux/rcupdate.h:274 ./include/linux/rcupdate.h:737 kernel/rcu/rcutorture.c:1443)
[   10.821960] rcu_torture_one_read (kernel/rcu/rcutorture.c:1590 kernel/rcu/rcutorture.c:1638)
[   10.821968] rcu_torture_reader (kernel/rcu/rcutorture.c:1730)
[   10.821976] kthread (kernel/kthread.c:321)
[   10.821986] ret_from_fork (arch/arm64/kernel/entry.S:1005)
[   10.821997] irq event stamp: 478635
[   10.822001] hardirqs last enabled at (478635): _raw_spin_unlock_irq (./arch/arm64/include/asm/irqflags.h:35 ./include/linux/spinlock_api_smp.h:168 kernel/locking/spinlock.c:202)
[   10.822016] hardirqs last disabled at (478634): __schedule (kernel/sched/core.c:5154 (discriminator 1))
[   10.822029] softirqs last enabled at (478626): __local_bh_enable_ip (./arch/arm64/include/asm/irqflags.h:85 kernel/softirq.c:262)
[   10.822040] softirqs last disabled at (478622): rcutorture_one_extend (./include/linux/bottom_half.h:19 kernel/rcu/rcutorture.c:1441)

Per this warning, softirqs cannot be disabled in a non-preemptible region
under CONFIG_PREEMPT_RT. Adjust RCU torture accordingly.

Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>
---
 kernel/rcu/rcutorture.c | 2 ++
 1 file changed, 2 insertions(+)

Comments

Valentin Schneider Aug. 4, 2021, 10:17 a.m. UTC | #1 
+Cc Scott, Sebastian

On 03/08/21 16:43, Paul E. McKenney wrote:
> On Tue, Aug 03, 2021 at 11:54:36PM +0100, Valentin Schneider wrote:

>> Running RCU torture with CONFIG_PREEMPT_RT under v5.13-rt1 triggers:

>>

>> [   10.821700] DEBUG_LOCKS_WARN_ON(this_cpu_read(softirq_ctrl.cnt))

>> [   10.821716] WARNING: CPU: 5 PID: 137 at kernel/softirq.c:173 __local_bh_disable_ip (kernel/softirq.c:173 (discriminator 31))

>> [   10.821739] Modules linked in:

>> [   10.821749] CPU: 5 PID: 137 Comm: rcu_torture_rea Not tainted 5.13.0-rt1-00005-g08bbda29766a #129

>> [   10.821759] Hardware name: ARM Juno development board (r0) (DT)

>> [   10.821765] pstate: 60000005 (nZCv daif -PAN -UAO -TCO BTYPE=--)

>> [   10.821938] Call trace:

>> [   10.821941] __local_bh_disable_ip (kernel/softirq.c:173 (discriminator 31))

>> [   10.821950] rcutorture_one_extend (./include/linux/rcupdate.h:274 ./include/linux/rcupdate.h:737 kernel/rcu/rcutorture.c:1443)

>> [   10.821960] rcu_torture_one_read (kernel/rcu/rcutorture.c:1590 kernel/rcu/rcutorture.c:1638)

>> [   10.821968] rcu_torture_reader (kernel/rcu/rcutorture.c:1730)

>> [   10.821976] kthread (kernel/kthread.c:321)

>> [   10.821986] ret_from_fork (arch/arm64/kernel/entry.S:1005)

>> [   10.821997] irq event stamp: 478635

>> [   10.822001] hardirqs last enabled at (478635): _raw_spin_unlock_irq (./arch/arm64/include/asm/irqflags.h:35 ./include/linux/spinlock_api_smp.h:168 kernel/locking/spinlock.c:202)

>> [   10.822016] hardirqs last disabled at (478634): __schedule (kernel/sched/core.c:5154 (discriminator 1))

>> [   10.822029] softirqs last enabled at (478626): __local_bh_enable_ip (./arch/arm64/include/asm/irqflags.h:85 kernel/softirq.c:262)

>> [   10.822040] softirqs last disabled at (478622): rcutorture_one_extend (./include/linux/bottom_half.h:19 kernel/rcu/rcutorture.c:1441)

>>

>> Per this warning, softirqs cannot be disabled in a non-preemptible region

>> under CONFIG_PREEMPT_RT. Adjust RCU torture accordingly.

>>

>> Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>

>> ---

>>  kernel/rcu/rcutorture.c | 2 ++

>>  1 file changed, 2 insertions(+)

>>

>> diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c

>> index 6096a7d14342..680f66b65f14 100644

>> --- a/kernel/rcu/rcutorture.c

>> +++ b/kernel/rcu/rcutorture.c

>> @@ -1537,6 +1537,8 @@ rcutorture_extend_mask(int oldmask, struct torture_random_state *trsp)

>>       * them on non-RT.

>>       */

>>      if (IS_ENABLED(CONFIG_PREEMPT_RT)) {

>

> This depends on some rcutorture patches in -rt that are not yet in

> -rcu.  Would -rt be a good place for this one, or are those patches

> now ready for -rcu?

>


Right, this goes along with

  72d6f4f680bf ("rcutorture: Avoid problematic critical section nesting on RT")

(from v5.13-rt1), which seems to apply cleanly on top of the current
mainline. So if we want this to go straight into -rcu, the above needs to
go along with it.

Thomas et al, any objections?

>                                                       Thanx, Paul

>

>> +		/* Can't disable bh in atomic context under PREEMPT_RT */

>> +		mask &= ~(RCUTORTURE_RDR_ATOM_BH | RCUTORTURE_RDR_ATOM_RBH);

>>              /*

>>               * Can't release the outermost rcu lock in an irq disabled

>>               * section without preemption also being disabled, if irqs

>> --

>> 2.25.1

>>
Paul E. McKenney Aug. 4, 2021, 10:37 p.m. UTC | #2 
On Wed, Aug 04, 2021 at 11:17:33AM +0100, Valentin Schneider wrote:
> 

> +Cc Scott, Sebastian

> 

> On 03/08/21 16:43, Paul E. McKenney wrote:

> > On Tue, Aug 03, 2021 at 11:54:36PM +0100, Valentin Schneider wrote:

> >> Running RCU torture with CONFIG_PREEMPT_RT under v5.13-rt1 triggers:

> >>

> >> [   10.821700] DEBUG_LOCKS_WARN_ON(this_cpu_read(softirq_ctrl.cnt))

> >> [   10.821716] WARNING: CPU: 5 PID: 137 at kernel/softirq.c:173 __local_bh_disable_ip (kernel/softirq.c:173 (discriminator 31))

> >> [   10.821739] Modules linked in:

> >> [   10.821749] CPU: 5 PID: 137 Comm: rcu_torture_rea Not tainted 5.13.0-rt1-00005-g08bbda29766a #129

> >> [   10.821759] Hardware name: ARM Juno development board (r0) (DT)

> >> [   10.821765] pstate: 60000005 (nZCv daif -PAN -UAO -TCO BTYPE=--)

> >> [   10.821938] Call trace:

> >> [   10.821941] __local_bh_disable_ip (kernel/softirq.c:173 (discriminator 31))

> >> [   10.821950] rcutorture_one_extend (./include/linux/rcupdate.h:274 ./include/linux/rcupdate.h:737 kernel/rcu/rcutorture.c:1443)

> >> [   10.821960] rcu_torture_one_read (kernel/rcu/rcutorture.c:1590 kernel/rcu/rcutorture.c:1638)

> >> [   10.821968] rcu_torture_reader (kernel/rcu/rcutorture.c:1730)

> >> [   10.821976] kthread (kernel/kthread.c:321)

> >> [   10.821986] ret_from_fork (arch/arm64/kernel/entry.S:1005)

> >> [   10.821997] irq event stamp: 478635

> >> [   10.822001] hardirqs last enabled at (478635): _raw_spin_unlock_irq (./arch/arm64/include/asm/irqflags.h:35 ./include/linux/spinlock_api_smp.h:168 kernel/locking/spinlock.c:202)

> >> [   10.822016] hardirqs last disabled at (478634): __schedule (kernel/sched/core.c:5154 (discriminator 1))

> >> [   10.822029] softirqs last enabled at (478626): __local_bh_enable_ip (./arch/arm64/include/asm/irqflags.h:85 kernel/softirq.c:262)

> >> [   10.822040] softirqs last disabled at (478622): rcutorture_one_extend (./include/linux/bottom_half.h:19 kernel/rcu/rcutorture.c:1441)

> >>

> >> Per this warning, softirqs cannot be disabled in a non-preemptible region

> >> under CONFIG_PREEMPT_RT. Adjust RCU torture accordingly.

> >>

> >> Signed-off-by: Valentin Schneider <valentin.schneider@arm.com>

> >> ---

> >>  kernel/rcu/rcutorture.c | 2 ++

> >>  1 file changed, 2 insertions(+)

> >>

> >> diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c

> >> index 6096a7d14342..680f66b65f14 100644

> >> --- a/kernel/rcu/rcutorture.c

> >> +++ b/kernel/rcu/rcutorture.c

> >> @@ -1537,6 +1537,8 @@ rcutorture_extend_mask(int oldmask, struct torture_random_state *trsp)

> >>       * them on non-RT.

> >>       */

> >>      if (IS_ENABLED(CONFIG_PREEMPT_RT)) {

> >

> > This depends on some rcutorture patches in -rt that are not yet in

> > -rcu.  Would -rt be a good place for this one, or are those patches

> > now ready for -rcu?

> >

> 

> Right, this goes along with

> 

>   72d6f4f680bf ("rcutorture: Avoid problematic critical section nesting on RT")

> 

> (from v5.13-rt1), which seems to apply cleanly on top of the current

> mainline. So if we want this to go straight into -rcu, the above needs to

> go along with it.


Ah, this one needs to have its changes conditioned on PREEMPT_RT.
The situations that this patch excludes really can happen in mainline,
so they still need to be tested in mainline.

It all comes back now.  ;-)

							Thanx, Paul

> Thomas et al, any objections?

> 

> >                                                       Thanx, Paul

> >

> >> +		/* Can't disable bh in atomic context under PREEMPT_RT */

> >> +		mask &= ~(RCUTORTURE_RDR_ATOM_BH | RCUTORTURE_RDR_ATOM_RBH);

> >>              /*

> >>               * Can't release the outermost rcu lock in an irq disabled

> >>               * section without preemption also being disabled, if irqs

> >> --

> >> 2.25.1

> >>
diff mbox series

Patch

diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
index 6096a7d14342..680f66b65f14 100644
--- a/kernel/rcu/rcutorture.c
+++ b/kernel/rcu/rcutorture.c
@@ -1537,6 +1537,8 @@  rcutorture_extend_mask(int oldmask, struct torture_random_state *trsp)
 	 * them on non-RT.
 	 */
 	if (IS_ENABLED(CONFIG_PREEMPT_RT)) {
+		/* Can't disable bh in atomic context under PREEMPT_RT */
+		mask &= ~(RCUTORTURE_RDR_ATOM_BH | RCUTORTURE_RDR_ATOM_RBH);
 		/*
 		 * Can't release the outermost rcu lock in an irq disabled
 		 * section without preemption also being disabled, if irqs