Mel Gorman
b795854b1f
sched/numa: Set preferred NUMA node based on number of private faults ...
Ideally it would be possible to distinguish between NUMA hinting faults that
are private to a task and those that are shared. If treated identically
there is a risk that shared pages bounce between nodes depending on
the order they are referenced by tasks. Ultimately what is desirable is
that task private pages remain local to the task while shared pages are
interleaved between sharing tasks running on different nodes to give good
average performance. This is further complicated by THP as even
applications that partition their data may not be partitioning on a huge
page boundary.
To start with, this patch assumes that multi-threaded or multi-process
applications partition their data and that in general the private accesses
are more important for cpu->memory locality in the general case. Also,
no new infrastructure is required to treat private pages properly but
interleaving for shared pages requires additional infrastructure.
To detect private accesses the pid of the last accessing task is required
but the storage requirements are a high. This patch borrows heavily from
Ingo Molnar's patch "numa, mm, sched: Implement last-CPU+PID hash tracking"
to encode some bits from the last accessing task in the page flags as
well as the node information. Collisions will occur but it is better than
just depending on the node information. Node information is then used to
determine if a page needs to migrate. The PID information is used to detect
private/shared accesses. The preferred NUMA node is selected based on where
the maximum number of approximately private faults were measured. Shared
faults are not taken into consideration for a few reasons.
First, if there are many tasks sharing the page then they'll all move
towards the same node. The node will be compute overloaded and then
scheduled away later only to bounce back again. Alternatively the shared
tasks would just bounce around nodes because the fault information is
effectively noise. Either way accounting for shared faults the same as
private faults can result in lower performance overall.
The second reason is based on a hypothetical workload that has a small
number of very important, heavily accessed private pages but a large shared
array. The shared array would dominate the number of faults and be selected
as a preferred node even though it's the wrong decision.
The third reason is that multiple threads in a process will race each
other to fault the shared page making the fault information unreliable.
Signed-off-by: Mel Gorman <mgorman@suse.de>
[ Fix complication error when !NUMA_BALANCING. ]
Reviewed-by: Rik van Riel <riel@redhat.com>
Cc: Andrea Arcangeli <aarcange@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Srikar Dronamraju <srikar@linux.vnet.ibm.com>
Signed-off-by: Peter Zijlstra <peterz@infradead.org>
Link: http://lkml.kernel.org/r/1381141781-10992-30-git-send-email-mgorman@suse.de
Signed-off-by: Ingo Molnar <mingo@kernel.org>
2013-10-09 12:40:35 +02:00
sched: Add NEED_RESCHED to the preempt_count
2013-09-25 14:07:49 +02:00
kgdb/sysrq: fix inconstistent help message of sysrq key
2013-04-30 17:04:10 -07:00
perf: Fix capabilities bitfield compatibility in 'struct perf_event_mmap_page'
2013-09-20 09:45:11 +02:00
kernel: replace strict_strto*() with kstrto*()
2013-09-12 15:38:03 -07:00
Remove GENERIC_HARDIRQ config option
2013-09-13 15:09:52 +02:00
PM / hibernate: Fix user space driven resume regression
2013-09-30 19:40:56 +02:00
TTY/Serial driver patches for 3.12-rc1
2013-09-03 11:38:36 -07:00
sched/numa: Set preferred NUMA node based on number of private faults
2013-10-09 12:40:35 +02:00
Merge branch 'timers-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2013-09-18 11:24:49 -05:00
Not much changes for the 3.12 merge window. The major tracing changes
2013-09-09 14:42:15 -07:00
kernel/hz.bc: ignore.
2013-04-22 07:09:06 -07:00
fs: Fix hang with BSD accounting on frozen filesystem
2013-05-04 14:57:58 -04:00
async: rename and redefine async_func_ptr
2013-03-12 13:59:14 -07:00
audit: fix endless wait in audit_log_start()
2013-09-24 17:00:26 -07:00
audit: fix mq_open and mq_unlink to add the MQ root as a hidden parent audit_names record
2013-07-09 10:33:19 -07:00
kernel/audit_tree.c:audit_add_tree_rule(): protect `rule' from kill_rules()
2013-06-12 16:29:46 -07:00
audit: Fix decimal constant description
2013-07-09 10:33:19 -07:00
audit: fix mq_open and mq_unlink to add the MQ root as a hidden parent audit_names record
2013-07-09 10:33:19 -07:00
xfs: update for v3.12-rc1
2013-09-09 11:19:09 -07:00
Kill indirect include of file.h from eventfd.h, use fdget() in cgroup.c
2013-09-07 19:54:57 -04:00
cgroup: make css_for_each_descendant() and friends include the origin css in the iteration
2013-08-08 20:11:27 -04:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/signal
2013-05-01 07:21:43 -07:00
proc: Supply PDE attribute setting accessor functions
2013-05-01 17:29:18 -04:00
Linux 3.12-rc4
2013-10-09 12:36:13 +02:00
ACPI / processor: Acquire writer lock to update CPU maps
2013-08-13 12:20:16 +02:00
Merge branch 'for-3.12' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
2013-09-03 18:25:03 -07:00
cputime: Use accessors to read task cputime stats
2013-01-27 19:23:31 +01:00
ptrace: revert "Prepare to fix racy accesses on task breakpoints"
2013-07-09 10:33:26 -07:00
extable: skip sorting if the table is empty
2013-09-11 15:58:25 -07:00
Revert "mm: sched: numa: Delay PTE scanning until a task is scheduled on a new node"
2013-10-09 12:40:17 +02:00
freezer: set PF_SUSPEND_TASK flag on tasks that call freeze_processes
2013-07-30 14:05:06 +02:00
futex: Use freezable blocking call
2013-06-25 23:11:19 +02:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/signal
2013-02-23 18:50:11 -08:00
userns: Kill nsown_capable it makes the wrong thing easy
2013-08-30 23:44:11 -07:00
kernel: delete __cpuinit usage from all core kernel files
2013-07-14 19:36:59 -04:00
hung_task debugging: Print more info when reporting the problem
2013-08-02 11:02:42 +02:00
Merge branch 'nohz/printk-v8' into irq/core
2013-02-05 00:48:46 +01:00
jump_label: Split jumplabel ratelimit
2013-08-09 07:53:54 -07:00
kernel: kallsyms: memory override issue, need check destination buffer length
2013-04-15 15:17:26 +09:30
locking: Fix copy/paste errors of "ARCH_INLINE_*_UNLOCK_BH"
2013-05-28 08:50:00 +02:00
kexec: remove unnecessary return
2013-09-11 15:59:10 -07:00
kernel/kmod.c: check for NULL in call_usermodehelper_exec()
2013-09-30 14:31:02 -07:00
kprobes: allow to specify custom allocator for insn caches
2013-09-11 15:58:52 -07:00
kernel: replace strict_strto*() with kstrto*()
2013-09-12 15:38:03 -07:00
kthread: implement probe_kthread_data()
2013-04-30 17:04:02 -07:00
lglock: Update lockdep annotations to report recursive local locks
2013-07-12 13:51:19 +02:00
lockdep: remove task argument from debug_check_no_locks_held
2013-05-12 14:16:21 +02:00
Remove GENERIC_HARDIRQ config option
2013-09-13 15:09:52 +02:00
CONFIG_SYMBOL_PREFIX: cleanup.
2013-03-15 15:09:43 +10:30
kernel/modsign_pubkey.c: fix init const for module signing code
2013-09-11 15:58:21 -07:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2013-09-05 08:50:26 -07:00
Merge branch 'core-locking-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2013-09-04 08:18:19 -07:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace
2013-09-07 14:35:32 -07:00
padata - Register hotcpu notifier after initialization
2013-08-29 14:37:59 +10:00
panic: call panic handlers before kmsg_dump
2013-09-11 15:59:30 -07:00
kernel/params: fix handling of signed integer types
2013-09-28 12:35:52 -07:00
pidns: fix free_pid() to handle the first fork failure
2013-09-30 14:31:03 -07:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace
2013-09-07 14:35:32 -07:00
posix_timers: fix racy timer delta caching on task exit
2013-07-03 16:54:42 +02:00
posix-timers: Remove unused variable
2013-04-18 12:51:19 +02:00
kernel: delete __cpuinit usage from all core kernel files
2013-07-14 19:36:59 -04:00
__ptrace_may_access() should not deny sub-threads
2013-09-11 15:59:01 -07:00
range: Do not add new blank slot with add_range_with_merge
2013-06-18 11:32:10 -05:00
rcu: Make call_rcu() leak callbacks for debug-object errors
2013-08-18 17:40:03 -07:00
Not much changes for the 3.12 merge window. The major tracing changes
2013-09-09 14:42:15 -07:00
rcu: Add const annotation to char * for RCU tracepoints and functions
2013-07-29 17:07:49 -04:00
rcu: Add const annotation to char * for RCU tracepoints and functions
2013-07-29 17:07:49 -04:00
rcu: Make rcutorture emit online failures if verbose
2013-08-20 11:38:45 -07:00
sched, rcu: Make RCU use resched_cpu()
2013-09-25 13:53:08 +02:00
nohz_full: Force RCU's grace-period kthreads onto timekeeping CPU
2013-08-31 14:44:02 -07:00
nohz_full: Force RCU's grace-period kthreads onto timekeeping CPU
2013-08-31 14:44:02 -07:00
rcutrace: single_open() leaks
2013-05-05 00:16:35 -04:00
kernel/reboot.c: re-enable the function of variable reboot_default
2013-09-24 17:00:26 -07:00
kernel: delete __cpuinit usage from all core kernel files
2013-07-14 19:36:59 -04:00
memcg: reduce function dereference
2013-09-12 15:38:02 -07:00
kernel/resource.c: remove the unneeded assignment in function __find_resource
2013-07-03 16:08:06 -07:00
sched/rt: Move rt specific bits into new header file
2013-02-07 20:51:08 +01:00
locking/rtmutex/tester: Set correct permissions on sysfs files
2013-04-10 14:48:37 +02:00
rtmutex: Document rt_mutex_adjust_prio_chain()
2013-05-28 09:23:52 +02:00
Revert "rw_semaphore: remove up/down_read_non_owner"
2013-03-23 15:53:52 -07:00
seccomp: allow BPF_XOR based ALU instructions.
2013-03-26 11:07:19 +11:00
semaphore: use `bool' type for semaphore_waiter's up
2013-04-30 17:04:08 -07:00
kernel-wide: fix missing validations on __get/__put/__copy_to/__copy_from_user()
2013-09-11 15:58:18 -07:00
kernel/smp.c: quit unconditionally enabling irqs in on_each_cpu_mask().
2013-09-11 15:58:25 -07:00
kernel: delete __cpuinit usage from all core kernel files
2013-07-14 19:36:59 -04:00
Linux 3.12-rc4
2013-10-09 12:36:13 +02:00
kernel/spinlock.c: add default arch_*_relax definitions for GENERIC_LOCKBREAK
2013-09-11 15:58:21 -07:00
srcu: use ACCESS_ONCE() to access sp->completed in srcu_read_lock()
2013-02-07 15:19:36 -08:00
stop_machine: Mark per cpu stopper enabled early
2013-02-26 22:25:17 +01:00
userns: Kill nsown_capable it makes the wrong thing easy
2013-08-30 23:44:11 -07:00
unify compat fanotify_mark(2), switch to COMPAT_SYSCALL_DEFINE
2013-05-09 13:46:38 -04:00
sched/numa: Favour moving tasks towards the preferred node
2013-10-09 12:40:26 +02:00
kernel: remove unnecessary head file
2013-06-26 18:01:46 +09:00
task_work: documentation
2013-09-11 15:58:27 -07:00
kernel/: rename random32() to prandom_u32()
2013-04-29 18:28:42 -07:00
sched: Rename sched.c as sched/core.c in comments and Documentation
2013-06-19 12:58:42 +02:00
kernel: Replace timeconst.pl with a bc script
2013-02-16 23:17:25 +01:00
sched: Introduce preempt_count accessor functions
2013-09-25 14:07:32 +02:00
Tracing updates for Linux 3.10
2013-04-29 13:55:38 -07:00
cputime: Use accessors to read task cputime stats
2013-01-27 19:23:31 +01:00
userns: Kill nsown_capable it makes the wrong thing easy
2013-08-30 23:44:11 -07:00
smp.h: move !SMP version of on_each_cpu() out-of-line
2013-09-11 15:58:25 -07:00
hlist: drop the node parameter from iterators
2013-02-27 19:10:24 -08:00
userns: Better restrictions on when proc and sysfs can be mounted
2013-08-26 19:17:03 -07:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace
2013-09-07 14:35:32 -07:00
userns: Kill nsown_capable it makes the wrong thing easy
2013-08-30 23:44:11 -07:00
kernel/utsname_sysctl.c: put get/get_uts() into CONFIG_PROC_SYSCTL code block
2013-02-27 19:10:22 -08:00
kernel: fix new kernel-doc warning in wait.c
2013-08-19 09:08:54 -07:00
watchdog: update watchdog_thresh properly
2013-09-24 17:00:25 -07:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial
2013-09-06 09:36:28 -07:00
sched: Rename sched.c as sched/core.c in comments and Documentation
2013-06-19 12:58:42 +02:00
b795854b1f