| Message ID | 20210202090010.5350-1-schwenderjonathan@gmail.com |
|---|---|
| State | New |
| Headers | show |
| Series | [RFC,v2] sched/rt: Fix RT (group) throttling with nohz_full | expand |
On 2021-02-02 10:00:10 [+0100], Jonathan Schwender wrote: > If nohz_full is enabled (more precisely HK_FLAG_TIMER is set), then > do_sched_rt_period_timer may be called on a housekeeping CPU, > which would not service the isolated CPU for a non-root cgroup > (requires a kernel with RT_GROUP_SCHEDULING). > This causes RT tasks in a non-root cgroup to get throttled > indefinitely (unless throttling is disabled) once the timer has > been moved to a housekeeping CPU. > To fix this, housekeeping CPUs now service all online CPUs > if HK_FLAG_TIMER (nohz_full) is set. This originates from https://lore.kernel.org/linux-rt-users/b07b6fc7-1a5f-0dcc-ca65-821de96cd8b4@gmail.com/ Could someone please take a look? Sebastian
diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index 8f720b71d13d..879abb5de023 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -866,9 +866,16 @@ static int do_sched_rt_period_timer(struct rt_bandwidth *rt_b, int overrun) * isolation is really required, the user will turn the throttle * off to kill the perturbations it causes anyway. Meanwhile, * this maintains functionality for boot and/or troubleshooting. + * If nohz_full is active and the timer was offloaded to a + * housekeeping CPU, sched_rt_period_mask() will not contain + * the isolated CPU. To prevent indefinite throttling of tasks + * on isolated CPUs, housekeeping CPUs service all online CPUs. */ - if (rt_b == &root_task_group.rt_bandwidth) + if (rt_b == &root_task_group.rt_bandwidth + || (housekeeping_enabled(HK_FLAG_TIMER) + && housekeeping_cpu(smp_processor_id(), HK_FLAG_TIMER))) { span = cpu_online_mask; + } #endif for_each_cpu(i, span) { int enqueue = 0;
RFC v2 changes: Fix compile error if CONFIG_SMP is not set, which was Reported-by: kernel test robot <lkp@intel.com> I'm now using smp_processor_id() to get the ID of the CPU the timer is running on. If nohz_full is enabled (more precisely HK_FLAG_TIMER is set), then do_sched_rt_period_timer may be called on a housekeeping CPU, which would not service the isolated CPU for a non-root cgroup (requires a kernel with RT_GROUP_SCHEDULING). This causes RT tasks in a non-root cgroup to get throttled indefinitely (unless throttling is disabled) once the timer has been moved to a housekeeping CPU. To fix this, housekeeping CPUs now service all online CPUs if HK_FLAG_TIMER (nohz_full) is set. I'm not really sure how this relates to Mike Galbraith previous commit e221d028bb08 ("sched,rt: fix isolated CPUs leaving root_task_group indefinitely throttled"), (which is dated before the housekeeping changes,) so I'm posting this as an RFC. Signed-off-by: Jonathan Schwender <schwenderjonathan@gmail.com> --- kernel/sched/rt.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)