Davide Libenzi
4ede816ac3
epoll keyed wakeups: add __wake_up_locked_key() and __wake_up_sync_key() ...
This patchset introduces wakeup hints for some of the most popular (from
epoll POV) devices, so that epoll code can avoid spurious wakeups on its
waiters.
The problem with epoll is that the callback-based wakeups do not, ATM,
carry any information about the events the wakeup is related to. So the
only choice epoll has (not being able to call f_op->poll() from inside the
callback), is to add the file* to a ready-list and resolve the real events
later on, at epoll_wait() (or its own f_op->poll()) time. This can cause
spurious wakeups, since the wake_up() itself might be for an event the
caller is not interested into.
The rate of these spurious wakeup can be pretty high in case of many
network sockets being monitored.
By allowing devices to report the events the wakeups refer to (at least
the two major classes - POLLIN/POLLOUT), we are able to spare useless
wakeups by proper handling inside the epoll's poll callback.
Epoll will have in any case to call f_op->poll() on the file* later on,
since the change to be done in order to have the full event set sent via
wakeup, is too invasive for the way our f_op->poll() system works (the
full event set is calculated inside the poll function - there are too many
of them to even start thinking the change - also poll/select would need
change too).
Epoll is changed in a way that both devices which send event hints, and
the ones that don't, are correctly handled. The former will gain some
efficiency though.
As a general rule for devices, would be to add an event mask by using
key-aware wakeup macros, when making up poll wait queues. I tested it
(together with the epoll's poll fix patch Andrew has in -mm) and wakeups
for the supported devices are correctly filtered.
Test program available here:
http://www.xmailserver.org/epoll_test.c
This patch:
Nothing revolutionary here. Just using the available "key" that our
wakeup core already support. The __wake_up_locked_key() was no brainer,
since both __wake_up_locked() and __wake_up_locked_key() are thin wrappers
around __wake_up_common().
The __wake_up_sync() function had a body, so the choice was between
borrowing the body for __wake_up_sync_key() and calling it from
__wake_up_sync(), or make an inline and calling it from both. I chose the
former since in most archs it all resolves to "mov $0, REG; jmp ADDR".
Signed-off-by: Davide Libenzi <davidel@xmailserver.org>
Cc: Alan Cox <alan@lxorguk.ukuu.org.uk>
Cc: Ingo Molnar <mingo@elte.hu>
Cc: David Miller <davem@davemloft.net>
Cc: William Lee Irwin III <wli@movementarian.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2009-04-01 08:59:20 -07:00
PM: Introduce functions for suspending and resuming device interrupts
2009-03-30 21:46:54 +02:00
pm: rework includes, remove arch ifdefs
2009-04-01 08:59:16 -07:00
Merge branch 'timers-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2009-03-26 16:05:42 -07:00
Merge commit 'origin/master' into next
2009-03-11 17:10:07 +11:00
[CVE-2009-0029] System call wrappers part 04
2009-01-14 14:15:19 +01:00
async: remove the temporary (2.6.29) "async is off by default" code
2009-03-28 13:05:30 -07:00
[PATCH] fix broken timestamps in AVC generated by kernel threads
2008-12-09 02:27:41 -05:00
fixing audit rule ordering mess, part 1
2009-01-04 15:14:41 -05:00
audit: validate comparison operations, store them in sane form
2009-01-04 15:14:42 -05:00
audit: validate comparison operations, store them in sane form
2009-01-04 15:14:42 -05:00
make sure that filterkey of task,always rules is reported
2009-01-04 15:14:42 -05:00
[CVE-2009-0029] System call wrappers part 04
2009-01-14 14:15:19 +01:00
vfs: simple_set_mnt() should return void
2009-03-27 14:44:03 -04:00
cgroups: fix probable race with put_css_set[_taskexit] and find_css_set
2008-10-20 08:52:38 -07:00
freezer_cg: disable writing freezer.state of root cgroup
2008-11-12 17:17:16 -08:00
Allow times and time system calls to return small negative values
2009-01-06 15:59:13 -08:00
kernel/configs.c: remove useless comments
2008-10-20 08:52:34 -07:00
cpumask: use set_cpu_active in init/main.c
2009-03-30 22:05:12 +10:30
cpuset: fix possible deadlock in async_rebuild_sched_domains
2009-01-19 02:44:00 +01:00
CRED: Inaugurate COW credentials
2008-11-14 10:39:23 +11:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/security-testing-2.6
2009-01-09 13:59:25 -08:00
schedstat: consolidate per-task cpu runtime stats
2008-12-18 13:54:01 +01:00
dma-coherent: Restore dma_alloc_from_coherent() large alloc fall back policy.
2009-01-21 18:51:53 +09:00
kernel/dma.c: remove a CVS keyword
2008-10-16 11:21:30 -07:00
[CVE-2009-0029] System call wrappers part 04
2009-01-14 14:15:19 +01:00
Merge branch 'linus' into x86/apic
2009-02-13 09:44:22 +01:00
Merge branch 'core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2008-12-30 16:10:19 -08:00
cpumask: use mm_cpumask() wrapper: kernel/fork.c
2009-03-30 22:05:13 +10:30
freezer_cg: use thaw_process() in unfreeze_cgroup()
2008-10-30 11:38:45 -07:00
futex: remove the pointer math from double_unlock_hb, fix
2009-03-13 10:32:07 +01:00
CRED: Use RCU to access another task's creds and to release a task's own creds
2008-11-14 10:39:19 +11:00
hrtimer: prevent negative expiry value after clock_was_set()
2009-01-30 22:35:34 +01:00
timers: split process wide cpu clocks/timers
2009-02-05 13:04:33 +01:00
Revert "kbuild: strip generated symbols from *.ko"
2009-01-14 21:38:20 +01:00
container freezer: implement freezer cgroup subsystem
2008-10-20 08:52:34 -07:00
rcu: provide RCU options on non-preempt architectures too
2008-12-25 09:31:28 +01:00
kexec: Change kexec jump code ordering
2009-03-30 21:46:55 +02:00
cpumask: remove dangerous CPU_MASK_ALL_PTR, &CPU_MASK_ALL
2009-03-30 22:05:11 +10:30
kprobes: check CONFIG_FREEZER instead of CONFIG_PM
2009-01-16 14:32:17 -05:00
kernel/ksysfs.c:fix dependence on CONFIG_NET
2009-01-06 10:44:31 -08:00
cpumask: remove dangerous CPU_MASK_ALL_PTR, &CPU_MASK_ALL
2009-03-30 22:05:11 +10:30
sched, latencytop: incorporate review feedback from Andrew Morton
2009-02-11 10:18:04 +01:00
lockdep: fix deadlock in lockdep_trace_alloc
2009-03-30 23:19:24 +02:00
lockdep: get_user_chars() redo
2009-02-14 23:28:22 +01:00
lockstat: warn about disabled lock debugging
2009-02-14 23:28:28 +01:00
lockdep: move state bit definitions around
2009-02-14 23:27:59 +01:00
PM: fix build for CONFIG_PM unset
2009-02-21 14:17:17 -08:00
markers/tracpoints: fix non-modular build
2008-11-16 09:52:03 +01:00
Merge branch 'core/percpu' into percpu-cpumask-x86-for-linus-2
2009-03-27 17:28:43 +01:00
mutex: implement adaptive spinning
2009-01-14 18:09:02 +01:00
mutex: implement adaptive spinning
2009-01-14 18:09:02 +01:00
mutex: adaptive spinnning, performance tweaks
2009-01-14 19:03:54 +01:00
mutex: implement adaptive spinning
2009-01-14 18:09:02 +01:00
Merge commit 'v2.6.28-rc6' into core/debug
2008-11-26 08:22:50 +01:00
ns_cgroup: remove unused spinlock
2009-01-08 08:31:02 -08:00
User namespaces: set of cleanups (v2)
2008-11-24 18:57:41 -05:00
stackprotector: update make rules
2009-02-10 00:41:54 +01:00
Fix compile warning in kernel/params.c
2008-10-23 12:09:00 -07:00
pid: generalize task_active_pid_ns
2009-01-08 08:31:12 -08:00
posix timers: fix RLIMIT_CPU && fork()
2009-03-23 20:43:35 +01:00
[CVE-2009-0029] System call wrappers part 05
2009-01-14 14:15:20 +01:00
PM: Fix suspend_console and resume_console to use only one semaphore
2009-02-21 14:17:18 -08:00
profiling: fix broken profiling regression
2009-02-10 00:50:37 +01:00
[CVE-2009-0029] System call wrappers part 27
2009-01-14 14:15:29 +01:00
rcu: Teach RCU that idle task is not quiscent state at boot
2009-02-26 04:08:14 +01:00
rcu: Teach RCU that idle task is not quiscent state at boot
2009-02-26 04:08:14 +01:00
rcu: Teach RCU that idle task is not quiscent state at boot
2009-02-26 04:08:14 +01:00
"Tree RCU": scalable classic RCU implementation
2008-12-18 21:56:04 +01:00
cpumask: convert rcutorture.c
2009-03-30 22:05:16 +10:30
rcu: Teach RCU that idle task is not quiscent state at boot
2009-02-26 04:08:14 +01:00
"Tree RCU": scalable classic RCU implementation
2008-12-18 21:56:04 +01:00
timers: add mod_timer_pending()
2009-02-18 19:26:33 +01:00
memcg: memory cgroup resource counters for hierarchy
2009-01-08 08:31:05 -08:00
resources: fix parameter name and kernel-doc
2009-01-15 16:39:38 -08:00
epoll keyed wakeups: add __wake_up_locked_key() and __wake_up_sync_key()
2009-04-01 08:59:20 -07:00
sched_clock: cleanups
2009-02-26 21:56:07 +01:00
sched: fix section mismatch
2009-01-06 11:07:15 +01:00
cpumask: remove cpumask_t from core
2009-03-30 22:05:17 +10:30
sched: remove unused fields from struct rq
2009-03-24 23:16:51 +01:00
Merge branch 'sched/urgent'; commit 'v2.6.29-rc5' into sched/core
2009-02-15 21:15:16 +01:00
Merge branch 'locking-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2009-03-30 17:17:35 -07:00
sched: add CONFIG_SMP consistency
2008-10-22 10:01:52 +02:00
Merge branch 'core/percpu' into percpu-cpumask-x86-for-linus-2
2009-03-27 17:28:43 +01:00
sched: remove unused fields from struct rq
2009-03-24 23:16:51 +01:00
x86-64: seccomp: fix 32/64 syscall hole
2009-03-02 15:41:30 -08:00
fix ptrace slowness
2009-03-23 09:22:31 -07:00
generic-ipi: use per cpu data for single cpu ipi calls
2009-01-30 18:31:08 +01:00
Merge branch 'core/percpu' into percpu-cpumask-x86-for-linus-2
2009-03-27 17:28:43 +01:00
softlock: fix false panic which can occur if softlockup_thresh is reduced
2009-01-14 11:48:07 +01:00
stacktrace: provide save_stack_trace_tsk() weak alias
2008-12-25 11:44:43 +01:00
cpumask: remove cpumask_t from core
2009-03-30 22:05:17 +10:30
sched: don't allow setuid to succeed if the user does not have rt bandwidth
2009-02-27 11:11:53 +01:00
[CVE-2009-0029] Make sys_syslog a conditional system call
2009-01-14 14:15:16 +01:00
mm: fix proc_dointvec_userhz_jiffies "breakage"
2009-04-01 08:59:13 -07:00
net: add ARP notify option for devices
2009-02-01 01:04:33 -08:00
cpumask: convert rest of files in kernel/
2009-01-01 10:12:28 +10:30
kprobes: add tests for register_kprobes
2009-01-06 15:59:20 -08:00
[CVE-2009-0029] System call wrappers part 01
2009-01-14 14:15:18 +01:00
Merge branch 'locking-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2009-03-30 17:17:35 -07:00
markers/tracpoints: fix non-modular build
2008-11-16 09:52:03 +01:00
Fix fixpoint divide exception in acct_update_integrals
2009-03-09 08:13:35 -07:00
[CVE-2009-0029] System call wrappers part 19
2009-01-14 14:15:26 +01:00
smp_call_function_single(): be slightly less stupid, fix #2
2009-01-12 16:04:37 +01:00
Merge branch 'master' into next
2009-03-24 10:52:46 +11:00
Fix recursive lock in free_uid()/free_user_ns()
2009-02-27 16:26:21 -08:00
sysctl: simplify ->strategy
2008-10-16 11:21:47 -07:00
wait: prevent exclusive waiter starvation
2009-02-05 12:56:48 -08:00
cpumask: use new cpumask_ functions in core code.
2009-03-30 22:05:16 +10:30
4ede816ac3