kernel: Add API to mark IRQs and kthreads as performance critical

On devices with a CPU that contains heterogeneous cores (e.g.,
big.LITTLE), it can be beneficial to place some performance-critical
IRQs and kthreads onto the performance CPU cluster in order to improve
performance.

This commit adds the following APIs:
-kthread_run_perf_critical() to create and start a perf-critical kthread
-irq_set_perf_affinity() to mark an active IRQ as perf-critical
-IRQF_PERF_CRITICAL to schedule an IRQ and any threads it may have onto
 performance CPUs
-PF_PERF_CRITICAL to mark a process (mainly a kthread) as performance
 critical (this is used by kthread_run_perf_critical())

In order to accommodate this new API, the following changes are made:
-Performance-critical IRQs are distributed evenly among online CPUs
 available in cpu_perf_mask
-Performance-critical IRQs have their affinities reaffined upon exit
 from suspend (since the affinities are broken when non-boot CPUs are
 disabled)
-Performance-critical IRQs and their threads have their affinities reset
 upon entering suspend, so that upon immediate suspend exit (when only
 the boot CPU is online), interrupts can be processed and interrupt
 threads can be scheduled onto an online CPU (otherwise we'd hit a
 kernel BUG)
-__set_cpus_allowed_ptr() is modified to enforce a performance-critical
 kthread's affinity
-Perf-critical IRQs are marked with IRQD_AFFINITY_MANAGED so userspace
 can't mess with their affinity

Signed-off-by: Sultan Alsawaf <sultan@kerneltoast.com>

include/linux/kthread.h

@@ -48,6 +48,23 @@ struct task_struct *kthread_create_on_cpu(int (*threadfn)(void *data),
 	__k;								   \
 })
 
+/**
+ * kthread_run_perf_critical - create and wake a performance-critical thread.
+ *
+ * Same as kthread_create().
+ */
+#define kthread_run_perf_critical(threadfn, data, namefmt, ...)		   \
+({									   \
+	struct task_struct *__k						   \
+		= kthread_create(threadfn, data, namefmt, ## __VA_ARGS__); \
+	if (!IS_ERR(__k)) {						   \
+		__k->flags |= PF_PERF_CRITICAL;				   \
+		kthread_bind_mask(__k, cpu_perf_mask);			   \
+		wake_up_process(__k);					   \
+	}								   \
+	__k;								   \
+})
+
 void kthread_bind(struct task_struct *k, unsigned int cpu);
 void kthread_bind_mask(struct task_struct *k, const struct cpumask *mask);
 int kthread_stop(struct task_struct *k);