No description
  • C 98.2%
  • Assembly 1%
  • Makefile 0.3%
  • Shell 0.2%
  • Python 0.1%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Dave Chinner 7a8f95bfb9 xfs: Lower CIL flush limit for large logs
commit 108a42358a05312b2128533c6462a3fdeb410bdf upstream.

The current CIL size aggregation limit is 1/8th the log size. This
means for large logs we might be aggregating at least 250MB of dirty objects
in memory before the CIL is flushed to the journal. With CIL shadow
buffers sitting around, this means the CIL is often consuming >500MB
of temporary memory that is all allocated under GFP_NOFS conditions.

Flushing the CIL can take some time to do if there is other IO
ongoing, and can introduce substantial log force latency by itself.
It also pins the memory until the objects are in the AIL and can be
written back and reclaimed by shrinkers. Hence this threshold also
tends to determine the minimum amount of memory XFS can operate in
under heavy modification without triggering the OOM killer.

Modify the CIL space limit to prevent such huge amounts of pinned
metadata from aggregating. We can have 2MB of log IO in flight at
once, so limit aggregation to 16x this size. This threshold was
chosen as it little impact on performance (on 16-way fsmark) or log
traffic but pins a lot less memory on large logs especially under
heavy memory pressure.  An aggregation limit of 8x had 5-10%
performance degradation and a 50% increase in log throughput for
the same workload, so clearly that was too small for highly
concurrent workloads on large logs.

This was found via trace analysis of AIL behaviour. e.g. insertion
from a single CIL flush:

xfs_ail_insert: old lsn 0/0 new lsn 1/3033090 type XFS_LI_INODE flags IN_AIL

$ grep xfs_ail_insert /mnt/scratch/s.t |grep "new lsn 1/3033090" |wc -l
1721823
$

So there were 1.7 million objects inserted into the AIL from this
CIL checkpoint, the first at 2323.392108, the last at 2325.667566 which
was the end of the trace (i.e. it hadn't finished). Clearly a major
problem.

Signed-off-by: Dave Chinner <dchinner@redhat.com>
Reviewed-by: Brian Foster <bfoster@redhat.com>
Reviewed-by: Allison Collins <allison.henderson@oracle.com>
Reviewed-by: Darrick J. Wong <darrick.wong@oracle.com>
Signed-off-by: Darrick J. Wong <darrick.wong@oracle.com>
Acked-by: Darrick J. Wong <djwong@kernel.org>
Signed-off-by: Chandan Babu R <chandan.babu@oracle.com>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2022-10-29 10:20:34 +02:00
arch ARM: dts: imx6sx: add missing properties for sram 2022-10-26 13:22:55 +02:00
block blk-mq: don't create hctx debugfs dir until q->debugfs_dir is created 2022-08-25 11:17:36 +02:00
certs certs/blacklist_hashes.c: fix const confusion in certs blacklist 2022-06-22 14:11:22 +02:00
crypto crypto: akcipher - default implementation for setting a private key 2022-10-26 13:22:45 +02:00
Documentation iio: ABI: Fix wrong format of differential capacitance channel ABI. 2022-10-26 13:22:34 +02:00
drivers thermal: intel_powerclamp: Use first online CPU as control_cpu 2022-10-26 13:23:01 +02:00
fs xfs: Lower CIL flush limit for large logs 2022-10-29 10:20:34 +02:00
include inet: fully convert sk->sk_rx_dst to RCU rules 2022-10-26 13:23:01 +02:00
init random: handle latent entropy and command line from random_init() 2022-06-22 14:11:17 +02:00
ipc ipc/mqueue: use get_tree_nodev() in mqueue_get_tree() 2022-06-14 18:11:41 +02:00
kernel cgroup/cpuset: Enable update_tasks_cpumask() on top_cpuset 2022-10-26 13:22:45 +02:00
lib dyndbg: let query-modname override actual module name 2022-10-26 13:22:37 +02:00
LICENSES
mm mm: pagewalk: Fix race between unmap and page walker 2022-10-15 07:54:36 +02:00
net inet: fully convert sk->sk_rx_dst to RCU rules 2022-10-26 13:23:01 +02:00
samples samples/kretprobes: Fix return value if register_kretprobe() failed 2021-11-17 09:48:39 +01:00
scripts kbuild: remove the target in signal traps when interrupted 2022-10-26 13:22:46 +02:00
security efi: Correct Macmini DMI match in uefi cert quirk 2022-10-15 07:54:39 +02:00
sound ALSA: hda/hdmi: Don't skip notification handling during PM operation 2022-10-26 13:22:31 +02:00
tools perf intel-pt: Fix segfault in intel_pt_print_info() with uClibc 2022-10-26 13:22:59 +02:00
usr initramfs: restore default compression behavior 2020-04-08 09:08:38 +02:00
virt KVM: Add infrastructure and macro to mark VM as bugged 2022-08-25 11:18:16 +02:00
.clang-format
.cocciconfig
.get_maintainer.ignore
.gitattributes
.gitignore
.mailmap
COPYING
CREDITS
Kbuild
Kconfig
MAINTAINERS MAINTAINERS: add Chandan as xfs maintainer for 5.4.y 2022-09-28 11:03:58 +02:00
Makefile Linux 5.4.220 2022-10-26 13:23:01 +02:00
README

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.