diff options
| author | Paul E. McKenney <paul.mckenney@linaro.org> | 2012-11-27 16:52:32 +0530 |
|---|---|---|
| committer | Viresh Kumar <viresh.kumar@linaro.org> | 2013-02-11 10:40:20 +0530 |
| commit | b34d95f12cf74392e5c27d84ad0c3f5ac0ddd0c9 (patch) | |
| tree | 9f2c4a3c7046b28f0982e8f5db5f6da26755bdff | |
| parent | 0c56725f435f4fd68b6df24af53da5c856d50d69 (diff) | |
cpu: No more __stop_machine() in _cpu_down()hotplug-free-from-stop-machine-v3
The _cpu_down() function invoked as part of the CPU-hotplug offlining
process currently invokes __stop_machine(), which is slow and inflicts
substantial real-time latencies on the entire system. This patch
substitutes stop_cpus() for __stop_machine() in order to improve
both performance and real-time latency.
This is currently unsafe, because there are a number of uses of
preempt_disable() that are intended to block CPU-hotplug offlining.
These will be fixed by using get/put_online_cpus_stable_atomic(), but in the
meantime, this commit is one way to help locate them.
Signed-off-by: Paul E. McKenney <paul.mckenney@linaro.org>
Signed-off-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com>
[ Srivatsa: Refer to get/put_online_cpus_stable_atomic() in the changelog ]
Signed-off-by: Srivatsa S. Bhat <srivatsa.bhat@linux.vnet.ibm.com>
| -rw-r--r-- | kernel/cpu.c | 9 |
1 files changed, 7 insertions, 2 deletions
diff --git a/kernel/cpu.c b/kernel/cpu.c index ab33f1c28c5..8c297ea175f 100644 --- a/kernel/cpu.c +++ b/kernel/cpu.c @@ -352,15 +352,20 @@ static int __ref take_cpu_down(void *_param) { struct take_cpu_down_param *param = _param; int err, cpu = (long)(param->hcpu); + unsigned long flags; prepare_cpu_take_down(cpu); /* Ensure this CPU doesn't handle any more interrupts. */ + local_irq_save(flags); err = __cpu_disable(); - if (err < 0) + if (err < 0) { + local_irq_restore(flags); return err; + } cpu_notify(CPU_DYING | param->mod, param->hcpu); + local_irq_restore(flags); return 0; } @@ -393,7 +398,7 @@ static int __ref _cpu_down(unsigned int cpu, int tasks_frozen) } smpboot_park_threads(cpu); - err = __stop_machine(take_cpu_down, &tcd_param, cpumask_of(cpu)); + err = stop_cpus(cpumask_of(cpu), take_cpu_down, &tcd_param); if (err) { /* CPU didn't die: tell everyone. Can't complain. */ smpboot_unpark_threads(cpu); |