[v3] lib/raid6: Reduce high latency by using migrate instead of preempt

We found an abnormally high latency when executing modprobe raid6_pq, the
latency is greater than 1.2s when CONFIG_PREEMPT_VOLUNTARY=y, greater than
67ms when CONFIG_PREEMPT=y, and greater than 16ms when CONFIG_PREEMPT_RT=y.

How to reproduce:
 - Install cyclictest
     sudo apt install rt-tests
 - Run cyclictest example in one terminal
     sudo cyclictest -S -p 95 -d 0 -i 1000 -D 24h -m
 - Modprobe raid6_pq in another terminal
     sudo modprobe raid6_pq

This is caused by ksoftirqd fail to scheduled due to disable preemption,
this time is too long and unreasonable.

Reduce high latency by using migrate_disabl()/emigrate_enable() instead of
preempt_disable()/preempt_enable(), the latency won't greater than 100us.

This patch beneficial for CONFIG_PREEMPT=y or CONFIG_PREEMPT_RT=y, but no
effect for CONFIG_PREEMPT_VOLUNTARY=y.

Cc: stable@vger.kernel.org
Fixes: fe5cbc6e06c7 ("md/raid6 algorithms: delta syndrome functions")
Fixes: cc4589ebfae6 ("Rename raid6 files now they're in a 'raid6' directory.")
Link: https://lore.kernel.org/linux-raid/b06c5e3ef3413f12a2c2b2a241005af9@linux.dev/T/#t # v1
Signed-off-by: Yajun Deng <yajun.deng@linux.dev>
---
 lib/raid6/algos.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

On Fri, Dec 17, 2021 at 8:09 PM Sebastian Andrzej Siewior
<bigeasy@linutronix.de> wrote:
>
> On 2021-12-17 10:16:10 [+0800], Yajun Deng wrote:
> > We found an abnormally high latency when executing modprobe raid6_pq, the
> > latency is greater than 1.2s when CONFIG_PREEMPT_VOLUNTARY=y, greater than
> > 67ms when CONFIG_PREEMPT=y, and greater than 16ms when CONFIG_PREEMPT_RT=y.
> >
> > How to reproduce:
> >  - Install cyclictest
> >      sudo apt install rt-tests
> >  - Run cyclictest example in one terminal
> >      sudo cyclictest -S -p 95 -d 0 -i 1000 -D 24h -m
> >  - Modprobe raid6_pq in another terminal
> >      sudo modprobe raid6_pq
> >
> > This is caused by ksoftirqd fail to scheduled due to disable preemption,
> > this time is too long and unreasonable.
> >
> > Reduce high latency by using migrate_disabl()/emigrate_enable() instead of
> > preempt_disable()/preempt_enable(), the latency won't greater than 100us.
> >
> > This patch beneficial for CONFIG_PREEMPT=y or CONFIG_PREEMPT_RT=y, but no
> > effect for CONFIG_PREEMPT_VOLUNTARY=y.
>
> Why does it matter? This is only during boot-up/ module loading or do I
> miss something?
> The delay is a jiffy so it depends on CONFIG_HZ. You do benchmark for
> the best algorithm and if you get preempted during that period then your
> results may be wrong and you make a bad selection.
>
> You can either enable one algorithm and or disable
> CONFIG_RAID6_PQ_BENCHMARK. I don't see the need for this patch not to
> mention the stable tree.

Exactly. We should not touch this. I've just sent a verbose
explanation in the original report thread.

--nX

> Sebastian