tick-sched: stop clockevent device when no longer required

Clockevent device can now be switched to ONESHOT_STOPPED mode, to avoid getting
spurious interrupts on a tickless CPU.

Thomas suggested not to do this 'behind the scene' and make sure all of the core
code is aware of it.

Locations from where clockevent devices must be stopped are highlighted with '*'
in below log.

Relevant scenarios (for both NO_HZ_IDLE and NO_HZ_FULL):

1: Mode: NOHZ_MODE_LOWRES

- Go tickless: When expires is set to 'KTIME_MAX', we don't reprogram clockevent
  device from tick_nohz_stop_sched_tick(). But tick_nohz_handler() have already
  reprogrammed it for next tick (and will continue doing that for all spurious
  ticks).

  * Stop clockevent device by calling tick_stop_event().

- Last timer expired: ticks would be reevaluated from irq_exit() and above will
  work here as well.

2. Mode: NOHZ_MODE_HIGHRES

- Go tickless: We cancel 'tick-sched' hrtimer from tick_nohz_stop_sched_tick()
  when expires is set to KTIME_MAX. hrtimer_cancel() would finally call
  hrtimer_force_reprogram() and we reevaluate enqueued hrtimers to reprogram
  clockevent device there.

  Currently if 'expires == KTIME_MAX', we skip reprogramming clockevent device.

  * At this point, we are guaranteed that no timers/hrtimers are pending and so
  we should stop clockevent device.

- Last Hrtimer is removed: Above will fix this as well as we will again
  call hrtimer_cancel().

- Last Hrtimer is expired: Lets check if above will work here as well:

  On irq_exit() we call tick_nohz_stop_sched_tick(), which must reevaluate
  pending timers to check if tick must be disabled again. If yes, we call
  hrtimer_cancel(). But the tick-sched hrtimer was never re-queued on a tickless
  CPU and so we wouldn't reach hrtimer_force_reprogram().

  * Some special handling for this is added to hrtimer_interrupt(). We are
  already evaluating next event in hrtimer_interrupt() and can stop clockevent
  device if expires is equal to KTIME_MAX.

A new API is tick_stop_event() is implemented to stop events by changing mode to
ONESHOT_STOPPED. It would do a WARN() if we were not in ONESHOT mode earlier.

Signed-off-by: Viresh Kumar <viresh.kumar@linaro.org>
[
Backported from viresh tick/oneshot-stopped
Manually applied patch for file hrtimer.c
note: revisit this backport as tick_program_event()
missing KTIME_MAX if loop in 3.14, should have some dependent
patch in >3.14 kernel, perhaps need to backport them too..revisit..
]
Signed-off-by: Santosh Shukla <santosh.shukla@linaro.org>

5 files changed, 64 insertions, 4 deletions

diff --git a/include/linux/tick.h b/include/linux/tick.h
index adf58889b728..f9bc9791719e 100644
--- a/include/linux/tick.h
+++ b/include/linux/tick.h
@@ -81,6 +81,7 @@ extern struct tick_device *tick_get_device(int cpu);
 # ifdef CONFIG_HIGH_RES_TIMERS
 extern int tick_init_highres(void);
 extern int tick_program_event(ktime_t expires, int force);
+extern void tick_stop_event(void);
 extern void tick_restart_event(void);
 extern void tick_setup_sched_timer(void);
 # endif
diff --git a/kernel/hrtimer.c b/kernel/hrtimer.c
index 263b663fb242..8af69b688fe8 100644
--- a/kernel/hrtimer.c
+++ b/kernel/hrtimer.c
@@ -591,7 +591,22 @@ hrtimer_force_reprogram(struct hrtimer_cpu_base *cpu_base, int skip_equal)
 	if (cpu_base->hang_detected)
 		return;
 
-	tick_program_event(cpu_base->expires_next, 1);
+	if (cpu_base->expires_next.tv64 != KTIME_MAX) {
+		tick_program_event(cpu_base->expires_next, 1);
+	} else {
+		/*
+		 * Don't need clockevent device anymore, stop it.
+		 *
+		 * We reach here only for NOHZ_MODE_HIGHRES mode and we are
+		 * guaranteed that no timers/hrtimers are enqueued on this cpu.
+		 *
+		 * We will reach here:
+		 * - when tick-sched hrtimer is cancelled to enter into idle or
+		 *   full dynticks mode
+		 * - or a hrtimer is cancelled on a tickless cpu
+		 */
+		tick_stop_event();
+	}
 }
 
 /*
@@ -1377,9 +1392,34 @@ retry:
 	cpu_base->expires_next = expires_next;
 	raw_spin_unlock(&cpu_base->lock);
 
-	/* Reprogramming necessary ? */
-	if (expires_next.tv64 == KTIME_MAX ||
-	    !tick_program_event(expires_next, 0)) {
+	if (expires_next.tv64 == KTIME_MAX) {
+		cpu_base->hang_detected = 0;
+
+		/*
+		 * Don't need clockevent device anymore, stop it.
+		 *
+		 * We reach here only for NOHZ_MODE_HIGHRES and we are
+		 * guaranteed to not have any pending timers/hrtimers on this
+		 * cpu.  Most of the scenarios will be covered by similar code
+		 * present in hrtimer_force_reprogram(), as we always try to
+		 * evaluate tick requirement on idle/irq exit and cancel
+		 * tick-sched hrtimer when tick isn't required anymore.
+		 *
+		 * It is required here as well for the extreme corner case.
+		 *
+		 * Suppose last hrtimer fires on a tickless CPU, we reach
+		 * tick_nohz_irq_exit(). While reevaluating we will get expires
+		 * == KTIME_MAX and try to cancel tick-sched hrtimer. BUT, the
+		 * tick-sched hrtimer was never queued on a tickless CPU and so
+		 * we wouldn't reach hrtimer_force_reprogram().
+		 *
+		 * So, disable Clockevent device here as well.
+		 */
+		tick_stop_event();
+		return;
+	}
+
+	if (!tick_program_event(expires_next, 0)) {
 		cpu_base->hang_detected = 0;
 		return;
 	}
diff --git a/kernel/time/tick-internal.h b/kernel/time/tick-internal.h
index d06c55945922..d3275fb0b7be 100644
--- a/kernel/time/tick-internal.h
+++ b/kernel/time/tick-internal.h
@@ -41,6 +41,7 @@ extern void tick_setup_oneshot(struct clock_event_device *newdev,
 			       void (*handler)(struct clock_event_device *),
 			       ktime_t nextevt);
 extern int tick_program_event(ktime_t expires, int force);
+extern void tick_stop_event(void);
 extern void tick_restart_event(void);
 extern void tick_oneshot_notify(void);
 extern int tick_switch_to_oneshot(void (*handler)(struct clock_event_device *));
diff --git a/kernel/time/tick-oneshot.c b/kernel/time/tick-oneshot.c
index 583d7c4fd555..5e156f17f4ec 100644
--- a/kernel/time/tick-oneshot.c
+++ b/kernel/time/tick-oneshot.c
@@ -22,6 +22,22 @@
 #include "tick-internal.h"
 
 /**
+ * tick_stop_event
+ */
+void tick_stop_event(void)
+{
+	struct clock_event_device *dev = __this_cpu_read(tick_cpu_device.evtdev);
+
+	/* Must be in ONESHOT or ONESHOT_STOPPED mode earlier */
+	WARN_ONCE(dev->mode != CLOCK_EVT_MODE_ONESHOT &&
+		  dev->mode != CLOCK_EVT_MODE_ONESHOT_STOPPED,
+		  "Current mode: %d\n", dev->mode);
+
+	/* stop clock event device */
+	clockevents_set_mode(dev, CLOCK_EVT_MODE_ONESHOT_STOPPED);
+}
+
+/**
  * tick_restart_event
  */
 void tick_restart_event(void)
diff --git a/kernel/time/tick-sched.c b/kernel/time/tick-sched.c
index 63cfd79f054b..3056db403696 100644
--- a/kernel/time/tick-sched.c
+++ b/kernel/time/tick-sched.c
@@ -652,6 +652,8 @@ static ktime_t tick_nohz_stop_sched_tick(struct tick_sched *ts,
 		 if (unlikely(expires.tv64 == KTIME_MAX)) {
 			if (ts->nohz_mode == NOHZ_MODE_HIGHRES)
 				hrtimer_cancel(&ts->sched_timer);
+			else
+				tick_stop_event();
 			goto out;
 		}