Kentaro Makita
da3bbdd463
fix soft lock up at NFS mount via per-SB LRU-list of unused dentries ...
[Summary]
Split LRU-list of unused dentries to one per superblock to avoid soft
lock up during NFS mounts and remounting of any filesystem.
Previously I posted here:
http://lkml.org/lkml/2008/3/5/590
[Descriptions]
- background
dentry_unused is a list of dentries which are not referenced.
dentry_unused grows up when references on directories or files are
released. This list can be very long if there is huge free memory.
- the problem
When shrink_dcache_sb() is called, it scans all dentry_unused linearly
under spin_lock(), and if dentry->d_sb is differnt from given
superblock, scan next dentry. This scan costs very much if there are
many entries, and very ineffective if there are many superblocks.
IOW, When we need to shrink unused dentries on one dentry, but scans
unused dentries on all superblocks in the system. For example, we scan
500 dentries to unmount a filesystem, but scans 1,000,000 or more unused
dentries on other superblocks.
In our case , At mounting NFS*, shrink_dcache_sb() is called to shrink
unused dentries on NFS, but scans 100,000,000 unused dentries on
superblocks in the system such as local ext3 filesystems. I hear NFS
mounting took 1 min on some system in use.
* : NFS uses virtual filesystem in rpc layer, so NFS is affected by
this problem.
100,000,000 is possible number on large systems.
Per-superblock LRU of unused dentried can reduce the cost in
reasonable manner.
- How to fix
I found this problem is solved by David Chinner's "Per-superblock
unused dentry LRU lists V3"(1), so I rebase it and add some fix to
reclaim with fairness, which is in Andrew Morton's comments(2).
1) http://lkml.org/lkml/2006/5/25/318
2) http://lkml.org/lkml/2006/5/25/320
Split LRU-list of unused dentries to each superblocks. Then, NFS
mounting will check dentries under a superblock instead of all. But
this spliting will break LRU of dentry-unused. So, I've attempted to
make reclaim unused dentrins with fairness by calculate number of
dentries to scan on this sb based on following way
number of dentries to scan on this sb =
count * (number of dentries on this sb / number of dentries in the machine)
- ToDo
- I have to measuring performance number and do stress tests.
- When unmount occurs during prune_dcache(), scanning on same
superblock, It is unable to reach next superblock because it is gone
away. We restart scannig superblock from first one, it causes
unfairness of reclaim unused dentries on first superblock. But I think
this happens very rarely.
- Test Results
Result on 6GB boxes with excessive unused dentries.
Without patch:
$ cat /proc/sys/fs/dentry-state
10181835 10180203 45 0 0 0
# mount -t nfs 10.124.60.70:/work/kernel-src nfs
real 0m1.830s
user 0m0.001s
sys 0m1.653s
With this patch:
$ cat /proc/sys/fs/dentry-state
10236610 10234751 45 0 0 0
# mount -t nfs 10.124.60.70:/work/kernel-src nfs
real 0m0.106s
user 0m0.002s
sys 0m0.032s
[akpm@linux-foundation.org: fix comments]
Signed-off-by: Kentaro Makita <k-makita@np.css.fujitsu.com>
Cc: Neil Brown <neilb@suse.de>
Cc: Trond Myklebust <trond.myklebust@fys.uio.no>
Cc: David Chinner <dgc@sgi.com>
Cc: "J. Bruce Fields" <bfields@fieldses.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2008-07-24 10:47:15 -07:00
i2c: New-style EEPROM driver using device IDs
2008-07-14 22:38:35 +02:00
lockd: Pass "struct sockaddr *" to new failover-by-IP function
2008-07-15 16:11:29 -04:00
Merge branch 'devel' of master.kernel.org:/home/rmk/linux-2.6-arm
2008-07-23 18:24:08 -07:00
IB/mlx4: Add support for blocking multicast loopback packets
2008-07-14 23:48:48 -07:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/drzeus/mmc
2008-07-16 15:17:52 -07:00
MTD: handle pci_name() being const
2008-07-21 21:55:03 -07:00
netfilter: nfnetlink_log: send complete hardware header
2008-07-21 10:11:00 -07:00
md: Protect access to mddev->disks list using RCU
2008-07-21 17:05:25 +10:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core-2.6
2008-07-22 13:13:47 -07:00
ssb: Include dma-mapping.h
2008-07-14 14:52:56 -04:00
Merge branch 'for-2.6.27' of git://linux-nfs.org/~bfields/linux
2008-07-20 21:21:46 -07:00
usb_serial: API all change
2008-07-22 13:03:22 -07:00
Merge branch 'linux-next' of git://git.kernel.org/pub/scm/linux/kernel/git/jbarnes/pci-2.6
2008-07-16 17:25:46 -07:00
async_tx: remove depend_tx from async_tx_sync_epilog
2008-07-17 17:59:55 -07:00
execve filename: document and export via auxiliary vector
2008-07-22 09:59:40 -07:00
block: Trivial fix for blk_integrity_rq()
2008-07-16 14:51:41 -07:00
mm: unexport __alloc_bootmem_core()
2008-07-24 10:47:14 -07:00
configfs: Allow ->make_item() and ->make_group() to return detailed errors.
2008-07-17 15:21:29 -07:00
Merge branch 'sched/for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2008-07-23 19:36:53 -07:00
cpu hotplug, sched: Introduce cpu_active_map and redo sched domain managment (take 2)
2008-07-18 13:22:25 +02:00
[SCSI] lib: Add support for the T10 (SCSI) Data Integrity Field CRC
2008-07-12 08:22:32 -05:00
crypto: hash - Move ahash functions into crypto/hash.h
2008-07-10 20:35:18 +08:00
tty: add more tty_port fields
2008-07-20 17:12:38 -07:00
I/OAT: I/OAT version 3.0 support
2008-07-22 17:30:57 -07:00
Merge branch 'linus' into core/rcu
2008-07-11 10:46:50 +02:00
dccp: Upgrade NDP count from 3 to 6 bytes
2008-07-13 11:51:40 +01:00
debugfs: Implement debugfs_remove_recursive()
2008-07-21 21:54:59 -07:00
dm: introduce merge_bvec_fn
2008-07-21 12:00:37 +01:00
driver core: remove DEVICE_ID_SIZE define
2008-07-21 21:54:53 -07:00
dm: introduce merge_bvec_fn
2008-07-21 12:00:37 +01:00
powerpc/cell: Add DMA_ATTR_WEAK_ORDERING dma attribute and use in Cell IOMMU code
2008-07-22 10:39:36 +10:00
async_tx: make async_tx_test_ack a boolean routine
2008-07-17 17:59:56 -07:00
dmaengine: Driver for the Synopsys DesignWare DMA controller
2008-07-08 11:59:42 -07:00
driver core: remove DEVICE_NAME_SIZE define
2008-07-21 21:54:53 -07:00
sysfs: add /sys/firmware/memmap
2008-07-08 17:55:41 +02:00
firmware: allow firmware files to be built into kernel image
2008-07-10 14:30:13 +01:00
Freezer: Introduce PF_FREEZER_NOSIG
2008-07-16 23:27:03 +02:00
fix soft lock up at NFS mount via per-SB LRU-list of unused dentries
2008-07-24 10:47:15 -07:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-2.6
2008-07-22 19:09:51 -07:00
gianfar: Add magic packet and suspend/resume support.
2008-07-16 17:57:47 -05:00
ftrace: add ftrace_kill_atomic
2008-07-11 15:49:21 +02:00
gs: use tty_port
2008-07-20 17:12:36 -07:00
page allocator: inline some __alloc_pages() wrappers
2008-07-24 10:47:14 -07:00
esp: use tty_port
2008-07-20 17:12:36 -07:00
HID: add n-trig digitizer usage
2008-07-23 15:25:21 +02:00
i2c-algo-pcf: Drop unused struct members
2008-07-14 22:38:31 +02:00
V4L/DVB (8246): tvaudio: Stop I2C driver ID abuse
2008-07-20 07:18:38 -03:00
i2c: Add detection capability to new-style drivers
2008-07-14 22:38:36 +02:00
ide: remove unused PC_FLAG_DRQ_INTERRUPT
2008-07-23 19:56:01 +02:00
mac80211: move QOS control helpers into ieee80211.h
2008-07-08 14:15:59 -04:00
packet: add PACKET_RESERVE sockopt
2008-07-18 18:05:19 -07:00
tun: Fix/rewrite packet filtering logic
2008-07-14 22:18:19 -07:00
vlan: Don't store VLAN tag in cb
2008-07-14 22:49:06 -07:00
ihex: request_ihex_firmware() function to load and validate firmware
2008-07-10 14:47:38 +01:00
NFS: Add string length argument to nfs_parse_server_address
2008-07-09 12:09:28 -04:00
ptrace children revamp
2008-07-16 18:02:33 -07:00
Merge master.kernel.org:/pub/scm/linux/kernel/git/torvalds/linux-2.6 into next
2008-07-21 00:55:14 -04:00
Merge branch 'genirq' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2008-07-15 10:39:22 -07:00
PNP: support optional IRQ resources
2008-07-16 23:27:07 +02:00
ipv6: __KERNEL__ ifdef struct ipv6_devconf
2008-07-21 13:41:16 -07:00
istallion: use tty_port
2008-07-20 17:12:37 -07:00
jbd2: Remove data=ordered mode support using jbd buffer heads
2008-07-11 19:27:31 -04:00
Merge branch 'linus' into core/printk
2008-07-10 08:17:14 +02:00
remove CONFIG_KMOD from core kernel code
2008-07-22 19:24:31 +10:00
kobject: reorder kobject to save space on 64 bit builds
2008-07-21 21:54:56 -07:00
KVM: Support mixed endian machines
2008-07-20 12:42:32 +03:00
KVM: MMU: nuke shadowed pgtable pages and ptes on memslot destruction
2008-07-20 12:42:40 +03:00
libata/ahci: enclosure management support
2008-07-14 15:59:33 -04:00
Merge branch 'auto-ftrace-next' into tracing/for-linus
2008-07-14 16:11:52 +02:00
mm: remove double indirection on tlb parameter to free_pgd_range() & Co
2008-07-24 10:47:15 -07:00
mm: Allow architectures to define additional protection bits
2008-07-09 16:30:45 +10:00
[S390] css: Use css_device_id for bus matching.
2008-07-14 10:02:12 +02:00
module: turn longs into ints for module sizes
2008-07-22 19:24:27 +10:00
vfs: add hooks for ext4's delayed allocation support
2008-07-11 19:27:31 -04:00
net: remove padding from struct socket on 64bit & increase objects/cache
2008-07-08 03:03:01 -07:00
net: Fix build failure with 'make mandocs'.
2008-07-22 14:09:06 -07:00
netfilter: cleanup netfilter_ipv6.h userspace header
2008-07-08 02:36:40 -07:00
NFS: use documenting macro constants for initializing ac{reg, dir}{min, max}
2008-07-09 12:09:40 -04:00
NFS: Move fs/nfs/iostat.h to include/linux
2008-07-09 12:09:17 -04:00
NFS: Allow redirtying of a completed unstable write.
2008-07-09 12:09:24 -04:00
NFS: Remove the redundant file_open entry from struct nfs_rpc_ops
2008-07-09 12:09:16 -04:00
of_gpio: Should use new <linux/gpio.h> header
2008-07-22 10:39:30 +10:00
Merge branch 'linux-next' of git://git.kernel.org/pub/scm/linux/kernel/git/jbarnes/pci-2.6
2008-07-16 17:25:46 -07:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/djbw/async_tx
2008-07-23 12:03:18 -07:00
PCI: Simplify PCI device PM code
2008-07-07 16:26:50 -07:00
percpu_counter: new function percpu_counter_sum_and_set
2008-07-11 19:27:31 -04:00
net-sched: cls_flow: add perturbation support
2008-07-14 20:36:32 -07:00
net_sched: Add size table for qdiscs
2008-07-20 00:08:47 -07:00
PCI: include linux/pm_wakeup.h for device_set_wakeup_capable
2008-07-14 14:30:21 -07:00
PNP: convert resource options to single linked list
2008-07-16 23:27:07 +02:00
netns: make get_proc_net() static
2008-07-22 14:19:19 -07:00
Security: split proc ptrace checking into read vs. attach
2008-07-14 15:01:47 +10:00
net_sched: Add size table for qdiscs
2008-07-20 00:08:47 -07:00
sg: reimplement sg mapping iterator
2008-07-23 14:42:09 +02:00
add a helper function to test if an object is on the stack
2008-07-24 10:47:14 -07:00
security: remove register_security hook
2008-07-14 15:04:06 +10:00
proc: consolidate per-net single-release callers
2008-07-18 04:07:44 -07:00
serial: use tty_port
2008-07-20 17:12:35 -07:00
Input: serio - trivial documentation fix
2008-07-08 10:33:01 -04:00
vlan: Don't store VLAN tag in cb
2008-07-14 22:49:06 -07:00
[NET] smc91x: prepare SMC_USE_PXA_DMA to be specified in platform data
2008-07-12 21:52:41 +01:00
net: Use standard structures for generic socket address structures.
2008-07-19 22:35:47 -07:00
stallion: use tty_port
2008-07-20 17:12:37 -07:00
move memory_read_from_buffer() from fs.h to string.h
2008-07-24 10:47:13 -07:00
synclink_gt: add serial bit order control
2008-07-22 13:03:29 -07:00
sysdev: Add utility functions for simple int/ulong variable sysdev attributes
2008-07-21 21:55:02 -07:00
driver core: Suppress sysfs warnings for device_rename().
2008-07-21 21:55:01 -07:00
tcp: Remove redundant checks when setting eff_sacks
2008-07-19 00:07:02 -07:00
textsearch: convert kmalloc + memset to kzalloc
2008-07-08 02:38:40 -07:00
tty: Split ldisc code into its own file
2008-07-22 13:03:27 -07:00
tty: rework break handling
2008-07-22 13:03:28 -07:00
tty: Ldisc revamp
2008-07-20 17:12:34 -07:00
UIO: minor style and comment fixes
2008-07-21 21:54:55 -07:00
USB: Force unbinding of drivers lacking reset_resume or other methods
2008-07-21 15:16:40 -07:00
USB: remove CVS keywords
2008-07-21 15:15:55 -07:00
V4L/DVB (8197): gspca: pac207 frames no more decoded in the subdriver.
2008-07-20 07:17:02 -03:00
mm/vmstat.c: proper externs
2008-07-24 10:47:14 -07:00
Merge branch 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4
2008-07-15 08:36:38 -07:00
xfrm: Add a XFRM_STATE_AF_UNSPEC flag to xfrm_usersa_info
2008-07-10 16:55:37 -07:00
