Nick Piggin
b3e19d924b
fs: scale mntget/mntput ...
The problem that this patch aims to fix is vfsmount refcounting scalability.
We need to take a reference on the vfsmount for every successful path lookup,
which often go to the same mount point.
The fundamental difficulty is that a "simple" reference count can never be made
scalable, because any time a reference is dropped, we must check whether that
was the last reference. To do that requires communication with all other CPUs
that may have taken a reference count.
We can make refcounts more scalable in a couple of ways, involving keeping
distributed counters, and checking for the global-zero condition less
frequently.
- check the global sum once every interval (this will delay zero detection
for some interval, so it's probably a showstopper for vfsmounts).
- keep a local count and only taking the global sum when local reaches 0 (this
is difficult for vfsmounts, because we can't hold preempt off for the life of
a reference, so a counter would need to be per-thread or tied strongly to a
particular CPU which requires more locking).
- keep a local difference of increments and decrements, which allows us to sum
the total difference and hence find the refcount when summing all CPUs. Then,
keep a single integer "long" refcount for slow and long lasting references,
and only take the global sum of local counters when the long refcount is 0.
This last scheme is what I implemented here. Attached mounts and process root
and working directory references are "long" references, and everything else is
a short reference.
This allows scalable vfsmount references during path walking over mounted
subtrees and unattached (lazy umounted) mounts with processes still running
in them.
This results in one fewer atomic op in the fastpath: mntget is now just a
per-CPU inc, rather than an atomic inc; and mntput just requires a spinlock
and non-atomic decrement in the common case. However code is otherwise bigger
and heavier, so single threaded performance is basically a wash.
Signed-off-by: Nick Piggin <npiggin@kernel.dk>
2011-01-07 17:50:33 +11:00
Merge branch 'next' of git://git.kernel.org/pub/scm/linux/kernel/git/djbw/async_tx
2010-10-27 19:04:36 -07:00
can: mcp251x: Don't use pdata->model for chip selection anymore
2010-10-18 15:16:08 +02:00
ceph: mark user pages dirty on direct-io reads
2010-12-17 09:54:40 -08:00
ceph: factor out libceph from Ceph file system
2010-10-20 15:37:28 -07:00
Input: adp5588-keys - unify common header defines
2010-11-03 10:44:20 -07:00
NLM: Fix a regression in lockd
2010-11-15 20:44:26 -05:00
ASoC: Fix off by one error in WM8994 EQ register bank size
2010-12-06 12:43:02 +00:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband
2010-10-26 17:54:22 -07:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/cjb/mmc
2010-11-27 07:16:29 +09:00
Merge git://git.infradead.org/mtd-2.6
2010-10-30 08:31:35 -07:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next-2.6
2010-10-23 11:47:02 -07:00
netfilter: install missing ebtables headers for userspace
2010-10-18 11:22:05 +02:00
regulator: Add option for machine drivers to enable the dummy regulator
2010-10-28 22:40:31 +01:00
Merge branch 'next-spi' of git://git.secretlab.ca/git/linux-2.6
2010-11-01 07:50:43 -04:00
SSB: Fix nvram_get on BCM47xx platform
2010-12-16 18:11:00 +00:00
nfsd4: fix 4.1 connection registration race
2010-11-02 17:13:52 -04:00
include/linux/unaligned: pack the whole struct rather than just the field
2010-12-22 19:43:34 -08:00
USB: musb: blackfin: push clkin value to platform resources
2010-11-05 14:13:32 +02:00
acpi: fix _OSI string setup regression
2010-12-11 01:28:58 -05:00
x86: Remove stale pmtimer_64.c
2010-10-15 21:18:59 +02:00
altera_uart: Fix missing prototype for registering an early console
2010-10-22 10:20:08 -07:00
[libata] support for > 512 byte sectors (e.g. 4K Native)
2010-10-21 20:21:03 -04:00
atm: correct sysfs 'device' link creation and parent relationships
2010-12-10 15:45:05 -08:00
atomic: add atomic_inc_not_zero_hint()
2010-11-12 07:55:30 -08:00
audit mmap
2010-10-30 08:45:43 -04:00
mm: declare some external symbols
2010-10-26 16:52:10 -07:00
gpio: add driver for basic memory-mapped GPIO controllers
2010-10-27 18:03:06 -07:00
netdev: bfin_mac: push settings to platform resources
2010-10-25 04:54:25 -04:00
exec: copy-and-paste the fixes into compat_do_execve() paths
2010-11-30 17:56:38 -08:00
block: remove REQ_HARDBARRIER
2010-11-10 14:54:09 +01:00
bit_spinlock: add required includes
2011-01-07 17:50:31 +11:00
block: remove REQ_HARDBARRIER
2010-11-10 14:54:09 +01:00
block: max hardware sectors limit wrapper
2010-12-17 08:36:01 +01:00
bootmem: Add alloc_bootmem_align()
2010-12-13 16:11:13 -08:00
fs: kill block_prepare_write
2010-10-25 21:18:20 -04:00
cgroup: add clone_children control file
2010-10-27 18:03:09 -07:00
clarify a usage constraint for cnt32_to_63()
2010-12-20 09:07:35 -08:00
Coda: add spin lock to protect accesses to struct coda_inode_info.
2010-10-25 08:02:40 -07:00
fs: provide rcu-walk aware permission i_ops
2011-01-07 17:50:29 +11:00
Coda: replace BKL with mutex
2010-10-25 08:02:40 -07:00
docbook: add more wait/wake/completion to device-drivers docbook
2010-10-26 17:32:41 -07:00
connector: remove lazy workqueue creation
2010-10-24 14:20:01 -07:00
driver core: prune docs about device_interface
2010-11-10 16:57:11 -08:00
fs: implement faster dentry memcmp
2011-01-07 17:50:32 +11:00
dccp: Extend CCID packet dequeueing interface
2010-10-28 10:27:00 -07:00
SYSFS: Allow boot time switching between deprecated and modern sysfs layout
2010-10-22 10:16:43 -07:00
dmaengine: provide dummy functions for DMA_ENGINE=n
2011-01-03 01:41:40 -08:00
dmar, x86: Use function stubs when CONFIG_INTR_REMAP is disabled
2010-11-26 09:57:36 +01:00
drbd: Silenced an assert
2010-10-22 15:55:22 +02:00
Merge branch 'for-2.6.37/drivers' of git://git.kernel.dk/linux-2.6-block
2010-10-22 17:03:12 -07:00
EDAC: Export edac sysfs class to users.
2010-10-21 14:47:59 +02:00
Revert "block: fix accounting bug on cross partition merges"
2010-10-24 22:06:02 +02:00
ethtool: Add support for vlan accleration.
2010-10-21 01:26:54 -07:00
nilfs2: make snapshots in checkpoint tree exportable
2010-10-23 09:24:34 +09:00
fanotify: split version into version and metadata_len
2010-12-15 13:56:33 -05:00
fbcmap: integer overflow bug
2010-11-17 14:55:45 +09:00
Update broken web addresses in the kernel.
2010-10-18 11:03:14 +02:00
fs: dcache per-bucket dcache hash locking
2011-01-07 17:50:31 +11:00
fs: fs_struct use seqlock
2011-01-07 17:50:27 +11:00
fsl-diu-fb: drop dead ioctl define
2010-11-15 13:49:51 +09:00
mxc_udc: add workaround for ENGcm09152 for i.MX35
2010-10-22 10:22:17 -07:00
fs: dcache remove dcache_lock
2011-01-07 17:50:23 +11:00
fs: dcache remove dcache_lock
2011-01-07 17:50:23 +11:00
fs: provide rcu-walk aware permission i_ops
2011-01-07 17:50:29 +11:00
Revert "block: fix accounting bug on cross partition merges"
2010-10-24 22:06:02 +02:00
PM / Hibernate: Fix memory corruption related to swap
2010-12-06 23:52:08 +01:00
hwmon: add generic GPIO fan driver
2010-10-25 14:11:37 -07:00
Input: add input driver for polled GPIO buttons
2010-12-06 17:15:15 -08:00
hardirq.h: needs sched.h if using BKL
2010-11-18 10:56:29 -08:00
Merge branch 'uc-logic' into for-linus
2010-10-23 22:49:32 +02:00
include/linux/highmem.h needs hardirq.h
2010-11-12 07:55:30 -08:00
perf,hw_breakpoint: Initialize hardware api earlier
2010-11-12 14:51:55 +01:00
i2c: Delete unused adapter IDs
2010-11-15 22:40:38 +01:00
i2c: Mark i2c_adapter.id as deprecated
2010-11-15 22:40:38 +01:00
docbook: add idr/ida to kernel-api docbook
2010-10-26 17:40:56 -07:00
Update broken web addresses in the kernel.
2010-10-18 11:03:14 +02:00
vlan: Add function to retrieve EtherType from vlan packets.
2010-11-12 12:30:57 -08:00
tproxy: added tproxy sockopt interface in the IPV6 layer
2010-10-21 16:08:28 +02:00
init.h: add some more documentation to __ref* tags
2010-10-22 10:21:26 -07:00
signals: move cred_guard_mutex from task_struct to signal_struct
2010-10-27 18:03:12 -07:00
Input: define separate EVIOCGKEYCODE_V2/EVIOCSKEYCODE_V2
2010-12-14 23:55:21 -08:00
Merge branch 'perf-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2010-10-27 18:48:00 -07:00
mm: stack based kmap_atomic()
2010-10-26 16:52:08 -07:00
block: remove unused copy_io_context()
2010-11-11 13:40:11 +01:00
resources: add arch hook for preventing allocation in reserved areas
2010-12-17 10:01:09 -08:00
tproxy: added tproxy sockopt interface in the IPV6 layer
2010-10-21 16:08:28 +02:00
genirq: Add single IRQ reservation helper
2010-10-26 10:33:27 +02:00
irq_work: Add generic hardirq context callbacks
2010-10-18 19:58:50 +02:00
genirq: Fix up irq_node() for irq_data changes.
2010-10-28 11:58:39 +02:00
jbd2: Fix I/O hang in jbd2_journal_release_jbd_inode
2010-10-27 21:25:12 -04:00
jump label: Add work around to i386 gcc asm goto bug
2010-10-29 14:45:29 -04:00
jump_label: Add atomic_t interface
2010-10-18 19:58:57 +02:00
fanotify: allow fanotify to be built
2010-10-28 17:22:13 -04:00
kdb,kgdb: fix sparse fixups
2010-10-22 15:34:12 -05:00
include/linux/kernel.h: Move logging bits to include/linux/printk.h
2010-11-15 13:37:37 -08:00
/proc/stat: fix scalability of irq sum of all cpu
2010-10-27 18:03:13 -07:00
kfifo: fix kfifo_alloc() to return a signed int value
2010-10-27 18:03:18 -07:00
debug_core,x86,blackfin: Clean up hw debug disable API
2010-10-29 13:14:41 -05:00
kobject: Introduce kset_find_obj_hinted.
2010-10-22 10:16:44 -07:00
kthread_work: make lockdep happy
2010-12-22 10:27:53 +01:00
KVM: PPC: Expose level based interrupt cap
2010-10-24 10:52:19 +02:00
Merge branch 'kvm-updates/2.6.37' of git://git.kernel.org/pub/scm/virt/kvm/kvm
2010-10-24 12:47:25 -07:00
KVM: Move kvm_guest_init out of generic code
2010-10-24 10:50:49 +02:00
leds: driver for National Semiconductor LP5521 chip
2010-11-12 07:55:32 -08:00
leds: driver for National Semiconductors LP5523 chip
2010-11-12 07:55:32 -08:00
led-class: always implement blinking
2010-11-12 07:55:32 -08:00
SCSI host lock push-down
2010-11-16 13:33:23 -08:00
hwmon: lis3: Short explanations of platform data fields
2010-10-25 14:11:39 -07:00
list.h: new helper - hlist_add_fake()
2010-10-25 21:24:15 -04:00
kernel: add bl_list
2011-01-07 17:50:31 +11:00
Merge branch 'irq-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2010-10-21 14:11:46 -07:00
9p: Use V9FS_MAGIC in statfs
2010-10-28 09:08:47 -05:00
phylib: Add support for Marvell 88E1149R devices.
2010-11-22 08:34:23 -08:00
div64_u64(): improve precision on 32bit platforms
2010-10-26 16:52:19 -07:00
Driver core: Add section count to memory_block struct
2010-10-22 10:16:44 -07:00
mem-hotplug: introduce {un}lock_memory_hotplug()
2010-12-02 14:51:15 -08:00
mm: wrap get_locked_pte() using __cond_lock()
2010-10-26 16:52:09 -07:00
oom: add per-mm oom disable count
2010-10-26 16:52:05 -07:00
mmu_notifier.h: fix comment spelling
2010-10-28 09:02:15 -07:00
writeback: do not sleep on the congestion queue if there are no congested BDIs or if significant congestion is not being encountered in the current zone
2010-10-26 16:52:07 -07:00
module: Update prototype for ref_module (formerly use_module)
2010-11-24 15:21:11 +10:30
modules: no need to align .modinfo strings
2010-10-26 16:52:13 -07:00
fs: scale mntget/mntput
2011-01-07 17:50:33 +11:00
Update broken web addresses in the kernel.
2010-10-18 11:03:14 +02:00
fs: rcu-walk for path lookup
2011-01-07 17:50:27 +11:00
fs: change d_compare for rcu-walk
2011-01-07 17:50:19 +11:00
fs: avoid inode RCU freeing for pseudo fs
2011-01-07 17:50:26 +11:00
net: Detect and ignore netif_stop_queue() calls before register_netdev()
2010-11-08 12:17:07 -08:00
netfilter: NF_HOOK_COND has wrong conditional
2010-11-12 08:26:06 +01:00
netlink: fix gcc -Wconversion compilation warning
2010-12-17 12:03:14 -08:00
bonding: Fix bonding drivers improper modification of netpoll structure
2010-10-18 08:32:07 -07:00
Merge branch 'for-2.6.37' of git://linux-nfs.org/~bfields/linux
2010-10-26 09:55:25 -07:00
fs: provide rcu-walk aware permission i_ops
2011-01-07 17:50:29 +11:00
NFSv4.1: pnfs: add LAYOUTGET and GETDEVICEINFO infrastructure
2010-10-24 18:07:10 -04:00
nfs: remove extraneous and problematic calls to nfs_clear_request
2010-12-07 23:02:44 -05:00
NFS: Ensure we return the dirent->d_type when it is known
2010-11-22 13:24:48 -05:00
nilfs2: change license of exported header file
2010-10-23 09:24:39 +09:00
driver core: prune docs about device_interface
2010-11-10 16:57:11 -08:00
of/device: Rework to use common platform_device_alloc() for allocating devices
2010-10-21 11:10:10 -06:00
of/flattree: Eliminate need to provide early_init_dt_scan_chosen_arch
2010-10-21 11:10:10 -06:00
of/irq: of_irq.c needs to include linux/irq.h
2010-10-21 11:34:55 -06:00
PM: Introduce library for device-specific OPPs (v7)
2010-10-17 01:57:50 +02:00
[S390] add support for nonquiescing sske
2010-10-25 16:10:15 +02:00
memcg: fix false positive VM_BUG on non-SMP
2010-11-25 06:50:40 +09:00
include/linux/pageblock-flags.h: fix set_pageblock_flags() macro definiton
2010-10-26 16:52:05 -07:00
mm: retry page fault when blocking on disk transfer
2010-10-26 16:52:09 -07:00
fs: scale mntget/mntput
2011-01-07 17:50:33 +11:00
PCI: Export some PCI PM functionality
2010-10-17 20:03:06 -07:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-2.6
2010-11-29 14:36:33 -08:00
PCI: add PCI_MSIX_TABLE/PBA defines
2010-10-17 20:03:03 -07:00
percpu: Remove the multi-page alignment facility
2010-10-27 17:53:25 +02:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/percpu
2010-10-22 17:31:36 -07:00
ext4: fix kernel oops if the journal superblock has a non-zero j_errno
2010-10-27 21:30:13 -04:00
perf: Fix duplicate events with multiple-pmu vs software events
2010-12-08 20:14:08 +01:00
phylib: make local function static
2010-10-24 15:07:11 -07:00
Un-inline get_pipe_info() helper function
2010-11-28 16:27:19 -08:00
driver core: platform_bus: allow runtime override of dev_pm_ops
2010-10-22 10:16:42 -07:00
PM / Runtime: Implement autosuspend support
2010-10-17 01:57:48 +02:00
PM / Runtime: Fix pm_runtime_suspended()
2010-12-16 17:12:25 +01:00
PM / Wakeup: Introduce wakeup source objects and event statistics (v3)
2010-10-17 01:57:43 +02:00
epoll: make epoll_wait() use the hrtimer range feature
2010-10-27 18:03:18 -07:00
fs: provide simple rcu-walk generic_check_acl implementation
2011-01-07 17:50:29 +11:00
include/linux/kernel.h: Move logging bits to include/linux/printk.h
2010-11-15 13:37:37 -08:00
ptrace: change signature of arch_ptrace()
2010-10-27 18:03:10 -07:00
backlight: add low threshold to pwm backlight
2010-11-12 07:55:33 -08:00
radix-tree: fix RCU bug
2010-11-12 07:55:32 -08:00
convert get_sb_nodev() users
2010-10-29 04:16:31 -04:00
ramoops: use the platform data structure instead of module params
2010-10-27 18:03:18 -07:00
printk: declare printk_ratelimit_state in ratelimit.h
2010-10-26 16:52:16 -07:00
kernel: add bl_list
2011-01-07 17:50:31 +11:00
BKL: remove extraneous #include <smp_lock.h>
2010-11-17 08:59:32 -08:00
fs: provide rcu-walk aware permission i_ops
2011-01-07 17:50:29 +11:00
include/linux/resource.h needs types.h
2010-11-12 07:55:30 -08:00
PM: Add sysfs attr for rechecking dev hash from PM trace
2010-10-17 01:57:50 +02:00
ring-buffer: Remove ring_buffer_event_time_delta()
2010-10-20 13:37:56 -04:00
rapidio: add handling of redundant routes
2010-10-27 18:03:16 -07:00
rapidio: add support for IDT CPS Gen2 switches
2010-10-27 18:03:16 -07:00
rapidio: fix IDLE2 bits corruption
2010-10-27 18:03:16 -07:00
rmap: wrap page_check_address() using __cond_lock()
2010-10-26 16:52:09 -07:00
net: rtnetlink.h -- only include linux/netdevice.h when used by the kernel
2010-11-15 11:29:30 -08:00
sched: Cure more NO_HZ load average woes
2010-12-08 20:15:04 +01:00
fs: rcu-walk for path lookup
2011-01-07 17:50:27 +11:00
vcs: invoke the vt update callback when /dev/vcs* is written to
2010-10-22 10:20:06 -07:00
secmark: make secmark object handling generic
2010-10-21 10:12:48 +11:00
semaphore: Remove mutex emulation
2010-10-30 12:12:50 +02:00
kernel: optimise seqlock
2011-01-07 17:50:27 +11:00
serial: abstraction for 8250 legacy ports
2010-10-22 10:20:10 -07:00
Merge branch 'omap-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tmlind/linux-omap-2.6
2010-10-25 13:46:56 -07:00
Input: serio - add support for PS2Mult multiplexer protocol
2010-10-18 09:33:31 -07:00
SFI: remove the v0.7 related definitions from sfi.h
2010-10-18 23:57:06 -04:00
sh: clkfwk: Disable init clk op for non-legacy clocks.
2010-11-15 18:48:25 +09:00
sh: intc: Fix up initializers for gcc 4.5.
2010-11-09 16:38:20 +09:00
ARM: shmobile: remove sh_timer_config clk member
2010-10-31 10:40:39 -04:00
net: avoid RCU for NOCACHE dst
2010-10-20 03:02:23 -07:00
kernel: kmem_ptr_validate considered harmful
2011-01-07 17:50:16 +11:00
Typedef SMP call function pointer
2010-10-27 17:28:36 +01:00
Fix build failure due to hwirq.h needing smp_lock.h
2010-11-17 14:58:36 -08:00
tcp: Replace time wait bucket msg by counter
2010-12-08 12:16:33 -08:00
net: Limit socket I/O iovec total length to INT_MAX.
2010-10-28 11:47:52 -07:00
PM: Allow wakeup events to abort freezing of tasks
2010-10-17 01:57:49 +02:00
mm: only build per-node scan_unevictable functions when NUMA is enabled
2010-10-26 16:52:05 -07:00
drivers/char/synclink_gt.c: add extended sync feature
2010-10-27 18:03:14 -07:00
ptrace: change signature of sys_ptrace() and friends
2010-10-27 18:03:10 -07:00
sysfs: Add sysfs_merge_group() and sysfs_unmerge_group()
2010-10-17 01:57:44 +02:00
taskstats: pad taskstats netlink response for aligment issues on ia64
2010-12-22 19:43:34 -08:00
Revert c6537d6742
2010-10-21 04:11:07 -07:00
signals: move cred_guard_mutex from task_struct to signal_struct
2010-10-27 18:03:12 -07:00
Merge branch 'tty-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty-2.6
2010-12-02 12:58:16 -08:00
tty: Make tiocgicount a handler
2010-10-22 10:20:04 -07:00
types.h: move misplaced comment
2010-10-26 16:52:03 -07:00
uio: Change mail address of Hans J. Koch
2010-11-10 16:57:11 -08:00
usb: musb: do not use dma for control transfers
2010-11-22 12:55:02 +02:00
usb-storage: add new no_read_capacity_16 quirk
2010-10-22 10:22:05 -07:00
[media] Add the via framebuffer camera controller driver
2010-10-21 13:45:28 -02:00
ACPI: video: fix build for VIDEO_OUTPUT_CONTROL=n
2010-12-11 02:01:35 -05:00
[media] V4L2: Add seek spacing and RDS CAP bits
2010-10-21 07:55:58 -02:00
Remove duplicate includes from many files
2010-10-27 18:03:18 -07:00
vmalloc: eagerly clear ptes on vunmap
2010-12-02 14:51:15 -08:00
workqueues: s/ON_STACK/ONSTACK/
2010-10-26 16:52:14 -07:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-unstable
2010-10-30 09:05:48 -07:00
