| Age | Commit message (Collapse) | Author |
|---|
|
lsk-v3.10-big.LITTLE
Conflicts:
kernel/irq/irqdesc.c
linaro/configs/big-LITTLE-MP.conf
|
|
This reverts commit f0862cb749c537c32de82912d30110098ddc6cd6. Analysis
from Vincent Guittot:
With the current implementation, the load average statistics of a sched entity
change according to other activity on the CPU even if this activity is done
between the running window of the sched entity and have no influence on the
running duration of the task.
When a task wakes up on the same CPU, we currently update last_runnable_update
with the return of __synchronize_entity_decay without updating the
runnable_avg_sum and runnable_avg_period accordingly. In fact, we have to sync
the load_contrib of the se with the rq's blocked_load_contrib before removing
it from the latter (with __synchronize_entity_decay) but we must keep
last_runnable_update unchanged for updating runnable_avg_sum/period during the
next update_entity_load_avg.
Signed-off-by: Mark Brown <broonie@linaro.org>
|
|
There is an error scenario where on a 1x1 HMP system (weight of the
hmp_slow_cpu_mask is 1) the short-cut of restricting the allowed cpu mask
of an rt tasks leads to triggering a kernel bug in the rt sched class
set_cpus_allowed function set_cpus_allowed_rt().
In case the task is on the run-queue and the weight of the required cpu mask
is 1 and this is different to the p->nr_cpus_allowed value, this back-end
function interprets this in such a way that a task changed from being
migratable to not migratable anymore and decrements the rt_nr_migratory
counter. There is a BUG_ON(!rq->rt.rt_nr_migratory) check in this code
path which triggers in this situation.
To circumvent this issue, set the number of allowed cpus for a task p to
the weight of the hmp_slow_cpu_mask before calling do_set_cpus_allowed()
in __setscheduler(). It will be set to this value in do_set_cpus_allowed()
after the call to the sched class related backend function any way. By
doing this, set_cpus_allowed_rt() returns without trying to update the
rt_nr_migratory counter.
This patch has been tested with a test device driver requiring a threaded
irq handler on a TC2 system with a reduced cpu mask (1 Cortex A15, 1
Cortex A7).
Signed-off-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
This patch limits the default affinity mask for all irqs to the cluster of
the little cpus.
This patch has the positive side effect that an irq thread which has its
IRQTF_RUNTHREAD set inside irq_thread() -> irq_wait_for_interrupt() will
not overwrite its struct task_struct->cpus_allowed with a full cpu mask of
desc->irq_data.affinity in irq_thread_check_affinity() essentially reverting
patch "HMP: experimental: Force all rt tasks to start on little domain."
for this irq thread.
Signed-off-by: Dietmar Eggemann <dietmar.eggemann@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
We use get_task_struct to increment the ref count on a task_struct
so that even if the task dies with a pending migration we are still
able to read the memory without causing a fault.
In the case of non-running tasks, we forgot to decrement the ref
count when we are done with the task.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
The scheduler will default packing to disabled, but this includes the
feature so that we can test it more easily.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
Since TC2 power curves don't really have a utilisation hotspot where
packing makes sense, if it is present for a TC2 system at least make
it default to disabled.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
The presence of packing permanently changed the idle balance
behaviour. Do not restrict idle balance on the smallest CPUs when
packing is present but disabled.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
If we migrate a sleeping task away from a CPU which has the
tick stopped, then both the clock_task and decay_counter will
be out of date for that CPU and we will not decay load correctly
regardless of how often we update the blocked load.
This is only an issue for tasks which are not on a runqueue
(because otherwise that CPU would be awake) and simultaneously
the CPU the task previously ran on has had the tick stopped.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
If an entity happens to sleep for less than one tick duration
the tracked load associated with that entity can be decayed by an
unexpectedly large amount if it is later migrated to a different
CPU. This can interfere with correct scheduling when entity load
is used for decision making.
The reason for this is that when an entity is dequeued and enqueued
quickly, such that se.avg.decay_count and cfs_rq.decay_counter
do not differ when that entity is enqueued again,
__synchronize_entity_decay skips the calculation step and also skips
clearing the decay_count. At a later time that entity may be
migrated and its load will be decayed incorrectly.
All users of this function expect decay_count to be zero'ed after
use.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
If we isolate CPUs, then we don't want random device interrupts on
them. Even w/o the user space irq balancer enabled we can end up with
irqs on non boot cpus.
Allow to restrict the default irq affinity mask.
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
This patch adds config fragments used to enable most of the features
used by big LITTLE MP.
This patch is the result of merging the following commits from the
Linaro Stable Kernel (LSK) 3.10 tree...
Commit 313c69010ed52a7bd52095708049e16c380c8e15
Commit 83952e566d1c41b2e0a70321e9d56cb8162f9a6d
Commit c4a782d985da03016e2b15aaef90bfc5cfe426f6
Commit 2732b0467a356e7cc08bf1e0f33481330335657a
Commit 7c88e2bd50fc093c96c912090b712b682ae72641
Commit 3231c8ce4b005f79ad27309907adc3de319e341c
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
At present we print per-entity load-tracking statistics for
cfs_rq of cgroups/runqueues. Given that per task statistics
is maintained, it can be used to know the contribution made
by the task to its parenting cfs_rq level.
This patch adds per-task load-tracking statistics to /proc/<PID>/sched.
Signed-off-by: Kamalesh Babulal <kamalesh@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20130625080336.GA20175@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
(cherry picked from commit 939fd731eb88a0cdd9058d0b0143563172a217d7)
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
At present we print per-entity load-tracking statistics for
cfs_rq of cgroups/runqueues. Given that per task statistics
is maintained, it can be used to know the contribution made
by the task to its parenting cfs_rq level.
This patch adds per-task load-tracking statistics to /proc/<PID>/sched.
Signed-off-by: Kamalesh Babulal <kamalesh@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/20130625080336.GA20175@linux.vnet.ibm.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
|
|
The woken migrated task will __synchronize_entity_decay(se); in
migrate_task_rq_fair, then it needs to set
`se->avg.last_runnable_update -= (-se->avg.decay_count) << 20' before
update_entity_load_avg, in order to avoid sleep time is updated twice
for se.avg.load_avg_contrib in both __syncchronize and
update_entity_load_avg.
However if the sleeping task is woken up from the same cpu, it miss
the last_runnable_update before update_entity_load_avg(se, 0, 1), then
the sleep time was used twice in both functions. So we need to remove
the double sleep time accounting.
Paul also contributed some code comments in this commit.
Signed-off-by: Alex Shi <alex.shi@intel.com>
Reviewed-by: Paul Turner <pjt@google.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1371694737-29336-5-git-send-email-alex.shi@intel.com
Signed-off-by: Ingo Molnar <mingo@kernel.org>
|
|
hmp_variable_scale_convert was used without guards in
__update_entity_runnable_avg. Guard it.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Mark Brown <broonie@linaro.org>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
hmp_variable_scale_convert was used without guards in
__update_entity_runnable_avg. Guard it.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Mark Brown <broonie@linaro.org>
|
|
lsk-v3.10-big.LITTLE
|
|
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
In order to allow userspace to restrict known low-load tasks to
little CPUs, we must export this knowledge from the kernel or
expect userspace to make their own attempts at figuring it out.
Since we now have a userspace requirement for an HMP implementation
to always have at least some sysfs files, change the integration
so that it only depends upon CONFIG_SCHED_HMP rather than
CONFIG_HMP_VARIABLE_SCALE. Fix Kconfig text to match.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
When migrating a runnable task, we use the CPU stopper on
the source CPU to ensure that the task to be moved is not
currently running. Before this patch, all forced migrations
(up, offload, idle pull) use the stopper for every migration.
Using the CPU stopper is mandatory only when a task is currently
running on a CPU. Otherwise tasks can be moved by locking the
source and destination run queues.
This patch checks to see if the task to be moved are currently
running. If not the task is moved directly without using the
stopper thread.
Signed-off-by: Mathieu Poirier <mathieu.poirier@linaro.org>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
Commit 2353c1f800 (arm: ipi raise/start/end tracing) added tracepoints for
IPIs in the generic GIC driver but only added definitions for them on ARM,
causing build failures on ARM64. Fix this by adding equivalent definitions
for arm64.
Signed-off-by: Mark Brown <broonie@linaro.org>
Acked-by: Liviu Dudau <Liviu.Dudau@arm.com>
|
|
Having the typedef in place for the tracepoints causes compiler crashes
in some situations. Just using void * directly avoids triggering the
issue and should have no effect on the trace.
Signed-off-by: Mark Brown <broonie@linaro.org>
Acked-by: Liviu Dudau <Liviu.Dudau@arm.com>
|
|
Commit 2353c1f800 (arm: ipi raise/start/end tracing) added tracepoints for
IPIs in the generic GIC driver but only added definitions for them on ARM,
causing build failures on ARM64. Fix this by adding equivalent definitions
for arm64.
Signed-off-by: Mark Brown <broonie@linaro.org>
Acked-by: Liviu Dudau <Liviu.Dudau@arm.com>
|
|
Having the typedef in place for the tracepoints causes compiler crashes
in some situations. Just using void * directly avoids triggering the
issue and should have no effect on the trace.
Signed-off-by: Mark Brown <broonie@linaro.org>
Acked-by: Liviu Dudau <Liviu.Dudau@arm.com>
|
|
lsk-v3.10-big.LITTLE
|
|
|
|
If we wake up a task on a little CPU, fill CPUs rather than
spread. Adds 2 new files to sys/kernel/hmp to control packing
behaviour.
packing_enable: task packing enabled (1) or disabled (0)
packing_limit: Runqueues will be filled up to this load ratio.
This functionality is disabled by default on TC2 as it lacks per-cpu
power gating so packing small tasks there doesn't make sense.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <Liviu.Dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
Accessing the task_struct can be racy in certain conditions, so
we need to only acquire the data when needed.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <Liviu.Dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
The previous API for hmp_up_migration reset the destination
CPU every time, regardless of if a migration was desired. The code
using it assumed that the value would not be changed unless
a migration was required. In one rare circumstance, this could
have lead to a task migrating to a little CPU at the wrong time.
Fixing that lead to a slight logical tweak to make the surrounding
APIs operate a bit more obviously.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Robin Randhawa <robin.randhawa@arm.com>
Signed-off-by: Liviu Dudau <Liviu.Dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
generic tracing for smp_cross_call function calls
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <Liviu.Dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
Add tracepoints for IPI raise events, and start and end of the
ipi handler.
Used to inspect the source of CPU wake-ups which are not already
traced - all other reasons for a CPU to wake-up are already
covered.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <Liviu.Dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
1. Replace magic numbers in code for migration trace.
Trace points still emit a number as force=<n> field:
force=0 : wakeup migration
force=1 : forced migration
force=2 : offload migration
force=3 : idle pull migration
2. Add trace to expose offload decision-making.
Also adds tracing rq->nr_running so that you can
look back to see what state the RQ was in at the time.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <Liviu.Dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
When the up-threshold is at 512 on TC2, behaviour looks OK since
the graphic-related tasks are very heavy due to lack of a GPU.
Increasing the up-threshold does not reduce power consumption.
When a GPU is present, graphic tasks are much less CPU-heavy and
so additional power may be saved by having a higher threshold.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <Liviu.Dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
lsk-v3.10-big.LITTLE
|
|
This merge is intended to tidyup the history of the big.LITTLE MP
patchset to enable ongoing maintenance of a standalone MP branch.
The only code change introduce by this merge is to add commit
0d5ddd14 (HMP: select 'best' task for migration rather than 'current')
This change is already in the Linaro Stable Kernel as commit
c5021c1eb9c73f38209180c65bd074ac70c97587
|
|
Prevents fork-migration adversely interacting with normal
migration (i.e. runqueues containing forked tasks being
selected as migration targets when there is a better
choice available)
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <liviu.dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
Avoids accesses through cfs_rq going bad when the cpu_rq doesn't
have a cfs member.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <liviu.dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
When an A15 goes idle, we should up-migrate anything which is
above the threshold and running on an A7.
Reuses the HMP force-migration spinlock, but adds its own new
cpu stopper client.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <liviu.dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
rq->nr_running was better than cfs.nr_running, since it includes
all tasks actually on the CPU. However, it includes RT tasks which
we would rather ignore at this point.
Switching to cfs.h_nr_running includes all the CFS tasks but no
RT tasks.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <liviu.dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
Experimentally, one of the best policies for HMP migration CPU
selection is to completely ignore part-loaded CPUs and only look
for idle ones. If there are no idle ones, we will choose the one
which was least-recently-disturbed.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <liviu.dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
The original intent here was to track unweighted runqueue load
with less resolution so we could use the least-recently-disturbed
runqueue to choose between 'closely related' load levels.
However, after experimenting with the resolution it turns out
that the following algorithm is highly beneficial for mobile
workloads.
In hmp_domain_min_load:
* If any CPU is zero, the overall load is zero
* If no CPUs are idle, the domain is 'fully loaded'
Additionally, the time since last migration count is used to
discriminate between idle CPUs.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <liviu.dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
Track when migrations were performed to runqueues.
Use this to decide between runqueues as migration targets when run
queues in an hmp domain have equal load.
Intention is to spread migration load amongst CPUs more fairly.
When all CPUs in an hmp domain are fully loaded, the existing code
always selects the last CPU as a migration target - this is unfair
and little better than doing no selection.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <liviu.dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
The hmp_get_{lightest,heaviest}_task() need to use
__pick_first_entity() to get a pointer to a sched_entity on the rq.
The current is not kept on the rq while running, so its rb-tree node
pointers are no longer valid.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <liviu.dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
When we are looking for a task to migrate up, select the heaviest
one in the first 5 runnable on the runqueue.
Likewise, when looking for a task to offload, select the lightest
one in the first 5 runnable on the runqueue.
Ensure task selected is runnable in the target domain.
This change is necessary in order to implement idle pull in a
sensible manner, but here is used in up-migration and offload to
select the correct target task.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <liviu.dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
It is sometimes desirable to run a kernel with HMP scheduling enabled
on a system which is not big.LITTLE, e.g. when building a multi-platform
kernel, or when testing a big.LITTLE system with one cluster disabled.
We should therefore allow for the situation where is no little domain.
Signed-off-by: Jon Medhurst <tixy@linaro.org>
Signed-off-by: Mark Brown <broonie@linaro.org>
|
|
lsk-v3.10-big.LITTLE
|
|
Prevents fork-migration adversely interacting with normal
migration (i.e. runqueues containing forked tasks being
selected as migration targets when there is a better
choice available)
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <liviu.dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
Avoids accesses through cfs_rq going bad when the cpu_rq doesn't
have a cfs member.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <liviu.dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
|
When an A15 goes idle, we should up-migrate anything which is
above the threshold and running on an A7.
Reuses the HMP force-migration spinlock, but adds its own new
cpu stopper client.
Signed-off-by: Chris Redpath <chris.redpath@arm.com>
Signed-off-by: Liviu Dudau <liviu.dudau@arm.com>
Signed-off-by: Jon Medhurst <tixy@linaro.org>
|
