jlelli / linux Goto Github PK

As done @

@lucabe72 shouldn't we wait the 0-lag time?

Implement hierarchical RT scheduling by nesting the SCHED_RT fixed-priority scheduler within SCHED_DEADLINE reservations, namely allowing for groups of tasks to be scheduled within a SCHED_DEADLINE reservation, choosing tasks within each group according to their RT priorities.

+-- SCHED_DL scheduler
   +-- SCHED_DL task 1 <rt=..., dl=..., period=...>
   +-- SCHED_DL task 2 <rt=..., dl=..., period=...>
   +-- ...
   +-- SCHED_RT group 1 <rt=..., period=...>
   |  +-- T1 <rtprio=...>
   |  +-- T2 <rtprio=...>
   +-- SCHED_RT group 2 <rt=..., period=...>
   |  +-- T3 <rtprio=...>
   |  +-- T4 <rtprio=...>
   ...

SCHED_DEADLINE reservation are attached to tasks by using sched_setattr syscall.
However, there is not yet a nice wrapper for it supported by glibc (similar to pthread_setschedparam for example).
Investigate is adding such a wrapper(s) is feasible.

A long time ago there was a patch [1] (written by Dario) adding DEADLINE bandwidth management control for task groups. That was then removed from the set of patches that made to mainline because outside of the bare minimum of features to possibly start playing with SCHED_DEADLINE, and because quite some discussion points remained open.
Fast forward to present day and more features have been added, DEADLINE usage is however still reserved to root only. Several things are still missing before we can comfortably relax privileges, bandwidth management for group of tasks being one of the most important (together with a better/safer PI mechanism I'd say).

Another (different) attempt to add cgroup support was proposed last year [2]. The set was implementing hierarchical scheduling support (RT entities running inside DEADLINE servers). Complexity (and maybe not enough documentation? :) made discussion around that proposal difficult to happen.
Even though hierarchical scheduling is still what we want in the end, this set tries to start getting there by adding cgroup based bandwidth management for SCHED_DEADLINE. The following design choices have been made (also detailed in changelog/doc):

• implementation is not hierarchical: only single/plain DEADLINE entities can be handled, and they get scheduled at root rq level
• DEADLINE_GROUP_SCHED requires RT_GROUP_SCHED (because of the points below)
• DEADLINE and RT share bandwidth; therefore, DEADLINE tasks will eat RT bandwidth, as they do today at root level; support for RT_RUNTIME_SHARE is however missing, an RT task might be able to exceed its group bandwidth constrain if such feature is enabled (more thinking required)
• and therefore cpu.rt_runtime_us and cpu.rt_period_us are still controlling a group bandwidth; however, two additional (read only) knobs are added:
  • cpu.dl_bw: maximum bandwidth available for the group on each CPU (rt_runtime_us/rt_period_us)
  • cpu.dl_total_bw : current total (across CPUs) amount of bandwidth allocated by the group (sum of tasks bandwidth)
• father/children/siblings rules are the same as for RT

Adding this kind of support should be useful to be able to let normal users use DEADLINE, as the sys admin (with root privileges) could reserve a fraction of the total available bandwidth to users and let
them allocate what needed inside such space.

SCHED_DEADLINE is currently only available to root users.
This however it's hindering adoption on applications (e.g., pro-audio) that run with normal privileges.
Main problem to safely relax privileges requirement is to implement a saner priority inheritance mechanism, as boosted threads are currently outside bandwidth enforcement.

Currently, partitioned EDF can be used only by disabling the admission control or by creating a root domain per CPU (and this will disable migrations for ALL tasks, not only deadline ones!).

We need some way to allow the usage of partitioned scheduling without breaking other tasks migrations or risking starvation (because we disabled admission control). Basically, per-runqueue admission control, with the constraint that deadline tasks must be affine to one single CPU.

Current implementation of deadline inheritance is broken (and also dangerous if potentially enabled for non-root users).
Problem is that boosting condition only checks absolute deadline of donor at the time rt_mutex_setprio() is executed and, after that, running task will use donor's parameters for its CBS. However, donor might have smaller bandwidth and longer relative deadline w.r.t. running task. Since running task is enqueued back with ENQUEUE_REPLENISH flag set, it will immediately use donor's parameters and by doing so it might be penalized (w.r.t. continue using its own parameters).
The potential danger for non-root usage comes for the fact that once boosted a task runs outside throttling mechanism (potentially starving lower prio tasks).