Linus Torvalds
35a891be96
xfs: reflink update for 4.9-rc1 ...
< XFS has gained super CoW powers! >
----------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
Included in this update:
- unshare range (FALLOC_FL_UNSHARE) support for fallocate
- copy-on-write extent size hints (FS_XFLAG_COWEXTSIZE) for fsxattr interface
- shared extent support for XFS
- copy-on-write support for shared extents
- copy_file_range support
- clone_file_range support (implements reflink)
- dedupe_file_range support
- defrag support for reverse mapping enabled filesystems
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAABAgAGBQJX/hrZAAoJEK3oKUf0dfodpwcQAKkTerNPhhDcthqWUJ2+jC7w
JIuhKUg2GYojJhIJ4+Ue1knmuBeIusda+PzGls+6gdy7GDGdux/esRIJSe1W7A5G
RNeumiSKVX5iYsZNUEX35O2a/SwUM1Sm5mcIFs4CxUwIRwE/cayNby6vrlVExvz7
Ns6YYOI2bldUHLsxedg8MLG0it1JGTADB9gwGgb98bxQ3bD/UBn3TF9xTlj+ZH22
ebnWsogSJOnUigOOSGeaQsmy1pJAhRIhvt+f481KuZak1pdQcK2feL4RcKw0NpNt
15LCYRqX6RexC684VYgJZxXB4EKyfS2Bma71q41A7dz1x36kw7+wG18xasBqU++p
GZwwL6si02rIGPMz1oD8xxZ0F97ADCGRmkgUHsCJKbP5UmGiP08K6GEN3osr5hAN
xAmn9AxcprXVnV3WmnFxpBeWY/qCEsvSQqJuKSThYqAilqUc8wN2u5g/eEpE6mmg
KEEhzaq5P4ovS/HOIQJWdBu1j5E9Mg2o/ncy87Q6uE+9Fa5AAP6GBWOtGcMwdFSU
adbN7dqjgoHMyNHFrmePqyJYtOZ2hZovDlVndxnYysl5ZBfiBEEDISmr+x6KcSlo
3kyOltYQLjEVu1sLOT3COCddn0jt5Lr1QhGeVepnrMlU2E1h4461viCNMDinJRIp
OYoMOS+J83G2FEFwgXYM
=Sa+Y
-----END PGP SIGNATURE-----
Merge tag 'xfs-reflink-for-linus-4.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/dgc/linux-xfs
< XFS has gained super CoW powers! >
----------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
Pull XFS support for shared data extents from Dave Chinner:
"This is the second part of the XFS updates for this merge cycle. This
pullreq contains the new shared data extents feature for XFS.
Given the complexity and size of this change I am expecting - like the
addition of reverse mapping last cycle - that there will be some
follow-up bug fixes and cleanups around the -rc3 stage for issues that
I'm sure will show up once the code hits a wider userbase.
What it is:
At the most basic level we are simply adding shared data extents to
XFS - i.e. a single extent on disk can now have multiple owners. To do
this we have to add new on-disk features to both track the shared
extents and the number of times they've been shared. This is done by
the new "refcount" btree that sits in every allocation group. When we
share or unshare an extent, this tree gets updated.
Along with this new tree, the reverse mapping tree needs to be updated
to track each owner or a shared extent. This also needs to be updated
ever share/unshare operation. These interactions at extent allocation
and freeing time have complex ordering and recovery constraints, so
there's a significant amount of new intent-based transaction code to
ensure that operations are performed atomically from both the runtime
and integrity/crash recovery perspectives.
We also need to break sharing when writes hit a shared extent - this
is where the new copy-on-write implementation comes in. We allocate
new storage and copy the original data along with the overwrite data
into the new location. We only do this for data as we don't share
metadata at all - each inode has it's own metadata that tracks the
shared data extents, the extents undergoing CoW and it's own private
extents.
Of course, being XFS, nothing is simple - we use delayed allocation
for CoW similar to how we use it for normal writes. ENOSPC is a
significant issue here - we build on the reservation code added in
4.8-rc1 with the reverse mapping feature to ensure we don't get
spurious ENOSPC issues part way through a CoW operation. These
mechanisms also help minimise fragmentation due to repeated CoW
operations. To further reduce fragmentation overhead, we've also
introduced a CoW extent size hint, which indicates how large a region
we should allocate when we execute a CoW operation.
With all this functionality in place, we can hook up .copy_file_range,
.clone_file_range and .dedupe_file_range and we gain all the
capabilities of reflink and other vfs provided functionality that
enable manipulation to shared extents. We also added a fallocate mode
that explicitly unshares a range of a file, which we implemented as an
explicit CoW of all the shared extents in a file.
As such, it's a huge chunk of new functionality with new on-disk
format features and internal infrastructure. It warns at mount time as
an experimental feature and that it may eat data (as we do with all
new on-disk features until they stabilise). We have not released
userspace suport for it yet - userspace support currently requires
download from Darrick's xfsprogs repo and build from source, so the
access to this feature is really developer/tester only at this point.
Initial userspace support will be released at the same time the kernel
with this code in it is released.
The new code causes 5-6 new failures with xfstests - these aren't
serious functional failures but things the output of tests changing
slightly due to perturbations in layouts, space usage, etc. OTOH,
we've added 150+ new tests to xfstests that specifically exercise this
new functionality so it's got far better test coverage than any
functionality we've previously added to XFS.
Darrick has done a pretty amazing job getting us to this stage, and
special mention also needs to go to Christoph (review, testing,
improvements and bug fixes) and Brian (caught several intricate bugs
during review) for the effort they've also put in.
Summary:
- unshare range (FALLOC_FL_UNSHARE) support for fallocate
- copy-on-write extent size hints (FS_XFLAG_COWEXTSIZE) for fsxattr
interface
- shared extent support for XFS
- copy-on-write support for shared extents
- copy_file_range support
- clone_file_range support (implements reflink)
- dedupe_file_range support
- defrag support for reverse mapping enabled filesystems"
* tag 'xfs-reflink-for-linus-4.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/dgc/linux-xfs: (71 commits)
xfs: convert COW blocks to real blocks before unwritten extent conversion
xfs: rework refcount cow recovery error handling
xfs: clear reflink flag if setting realtime flag
xfs: fix error initialization
xfs: fix label inaccuracies
xfs: remove isize check from unshare operation
xfs: reduce stack usage of _reflink_clear_inode_flag
xfs: check inode reflink flag before calling reflink functions
xfs: implement swapext for rmap filesystems
xfs: refactor swapext code
xfs: various swapext cleanups
xfs: recognize the reflink feature bit
xfs: simulate per-AG reservations being critically low
xfs: don't mix reflink and DAX mode for now
xfs: check for invalid inode reflink flags
xfs: set a default CoW extent size of 32 blocks
xfs: convert unwritten status of reverse mappings for shared files
xfs: use interval query for rmap alloc operations on shared files
xfs: add shared rmap map/unmap/convert log item types
xfs: increase log reservations for reflink
...
2016-10-13 20:28:22 -07:00
fbdev changes for 4.9
2016-10-12 11:01:37 -07:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next
2016-10-05 10:11:24 -07:00
can: dev: fix deadlock reported after bus-off
2016-09-22 10:01:21 +02:00
ceph: handle CEPH_SESSION_REJECT message
2016-10-03 16:13:50 +02:00
firmware /meson
treewide: remove redundant #include <linux/kconfig.h>
2016-10-11 15:06:33 -07:00
- Core Frameworks
2016-10-07 08:35:35 -07:00
Merge of primary rdma-core code for 4.9
2016-10-09 17:04:33 -07:00
MMC core:
2016-10-05 11:34:53 -07:00
Introduction of the MTD pairing scheme concept.
2016-10-08 20:56:54 -07:00
Merge branch 'smp-hotplug-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2016-10-03 19:43:08 -07:00
media updates for v4.9-rc1
2016-10-11 13:22:22 -07:00
qed: Add RoCE ll2 & GSI support
2016-10-03 23:22:47 -04:00
Merge tag 'md/4.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/shli/md
2016-10-07 09:45:43 -07:00
Merge remote-tracking branches 'regulator/topic/of', 'regulator/topic/pv88080', 'regulator/topic/rk808', 'regulator/topic/set-voltage' and 'regulator/topic/tps65218' into regulator-next
2016-09-30 09:13:58 -07:00
Merge remote-tracking branches 'spi/topic/fsl-espi', 'spi/topic/imx', 'spi/topic/jcore', 'spi/topic/loopback' and 'spi/topic/meson' into spi-next
2016-09-30 09:14:10 -07:00
TTY/Serial patches for 4.9-rc1
2016-10-03 20:11:49 -07:00
tty/serial: atmel: fix fractional baud rate computation
2016-09-27 12:54:03 +02:00
autofs4: move linux/auto_dev-ioctl.h to uapi/linux
2016-10-11 15:06:31 -07:00
autofs: move inclusion of linux/limits.h to uapi
2016-10-11 15:06:31 -07:00
block: export bio_free_pages to other modules
2016-09-22 07:48:03 -06:00
mm/vmalloc.c: fix align value calculation error
2016-10-07 18:46:26 -07:00
blkcg: Annotate blkg_hint correctly
2016-09-23 11:30:38 -06:00
Merge branch 'for-4.9/block-smp' of git://git.kernel.dk/linux-block
2016-10-09 17:32:20 -07:00
mm/nobootmem.c: remove duplicate macro ARCH_LOW_ADDRESS_LIMIT statements
2016-10-07 18:46:28 -07:00
bpf: allow access into map value arrays
2016-09-29 01:35:35 -04:00
bpf: allow access into map value arrays
2016-09-29 01:35:35 -04:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace
2016-10-06 09:52:23 -07:00
clk: change the type of clk_hw_onecell_data.num to unsigned int
2016-09-23 14:44:03 -07:00
mm, compaction: restrict full priority to non-costly orders
2016-10-07 18:46:29 -07:00
compat: remove compat_printk()
2016-09-27 21:20:53 -04:00
console: don't prefer first registered if DT specifies stdout-path
2016-10-07 18:46:30 -07:00
nmi_backtrace: generate one-line reports for idle cpus
2016-10-07 18:46:30 -07:00
xen: features and fixes for 4.9-rc0
2016-10-06 11:19:10 -07:00
cred: simpler, 1D supplementary groups
2016-10-07 18:46:30 -07:00
include/linux/ctype.h: make isdigit() table lookupless
2016-10-11 15:06:30 -07:00
dma-debug: add support for resource mappings
2016-09-26 22:16:41 +05:30
dma-mapping: introduce the DMA_ATTR_NO_WARN attribute
2016-10-11 15:06:32 -07:00
treewide: remove redundant #include <linux/kconfig.h>
2016-10-11 15:06:33 -07:00
f2fs: use crc and cp version to determine roll-forward recovery
2016-09-30 10:05:46 -07:00
vfs: add a FALLOC_FL_UNSHARE mode to fallocate to unshare a range of blocks
2016-10-03 09:11:14 -07:00
fs/file: more unsigned file descriptors
2016-09-27 18:47:38 -04:00
mm: split gfp_mask and mapping flags into separate fields
2016-10-11 15:06:34 -07:00
Lots of bug fixes and cleanups.
2016-10-07 15:15:33 -07:00
fanotify: use notification_lock instead of access_lock
2016-10-07 18:46:26 -07:00
This release cycle is rather small. Just a few fixes to tracing.
2016-10-06 11:48:41 -07:00
HID: add missing \n to end of dev_warn messages
2016-09-29 10:40:13 +02:00
thp: reduce usage of huge zero page's atomic counter
2016-10-07 18:46:28 -07:00
mm: remove unnecessary condition in remove_inode_hugepages
2016-10-07 18:46:29 -07:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next
2016-10-05 10:11:24 -07:00
i2c: export i2c_adapter_depth()
2016-09-24 10:48:18 +02:00
net: Update API for VF vlan protocol 802.1ad support
2016-09-24 08:01:26 -04:00
Fixing a bug in team driver due to incorrect 'unsigned int' to 'int' conversion
2016-10-07 23:50:39 -04:00
Merge branch 'iomap-4.9-dax' into for-next
2016-10-03 09:53:59 +11:00
ipv6 addrconf: implement RFC7559 router solicitation backoff
2016-09-30 01:54:28 -04:00
jiffies: add time comparison functions for 64 bit jiffies
2016-10-07 18:46:26 -07:00
Merge branch 'ras-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2016-10-03 13:22:39 -07:00
printk: reinstate KERN_CONT for printing continuation lines
2016-10-09 12:23:38 -07:00
min/max: remove sparse warnings when they're nested
2016-10-07 18:46:30 -07:00
kdump, vmcoreinfo: report memory sections virtual addresses
2016-10-11 15:06:33 -07:00
mm: kmemleak: avoid using __va() on addresses that don't have a lowmem mapping
2016-10-11 15:06:33 -07:00
kthread: add kerneldoc for kthread_create()
2016-10-11 15:06:33 -07:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next
2016-10-05 10:11:24 -07:00
libnvdimm, region: move region-mapping input-paramters to nd_mapping_desc
2016-09-30 19:13:42 -07:00
lockdep: make MAX_LOCKDEP_SUBCLASSES unconditionally visible
2016-09-24 10:48:32 +02:00
mcb: Add a dma_device to mcb_device
2016-09-27 12:33:47 +02:00
mm/memblock.c: expose total reserved memory
2016-10-07 18:46:28 -07:00
mm: memcontrol: consolidate cgroup socket tracking
2016-10-07 18:46:29 -07:00
linux/mm.h: canonicalize macro PAGE_ALIGNED() definition
2016-10-07 18:46:30 -07:00
kernel, oom: fix potential pgd_lock deadlock from __mmdrop
2016-10-07 18:46:27 -07:00
mnt: Add a per mount namespace limit on the number of mounts
2016-09-30 12:46:48 -05:00
ipmr, ip6mr: fix scheduling while atomic and a deadlock with ipmr_get_route
2016-09-25 23:41:39 -04:00
ipmr, ip6mr: fix scheduling while atomic and a deadlock with ipmr_get_route
2016-09-25 23:41:39 -04:00
libnvdimm, namespace: sort namespaces by dpa at init
2016-10-07 09:20:53 -07:00
Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next
2016-09-25 23:34:19 +02:00
Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next
2016-09-25 23:34:19 +02:00
netfilter: replace list_head with single linked list
2016-09-25 14:38:48 +02:00
nmi_backtrace: add more trigger_*_cpu_backtrace() methods
2016-10-07 18:46:30 -07:00
of/serial: move earlycon early_param handling to serial
2016-09-28 17:43:15 +02:00
gpio: OF: localize some gpiochip init functions
2016-10-03 23:38:11 +02:00
dmaengine/ARM: omap-dma: Fix the DMAengine compile test on non OMAP configs
2016-09-28 08:53:14 +05:30
mm, oom: enforce exit_oom_victim on current task
2016-10-07 18:46:28 -07:00
mm/page_owner: don't define fields on struct page_ext by hard-coding
2016-10-07 18:46:27 -07:00
mm/page_owner: move page_owner specific function to page_owner.c
2016-10-07 18:46:27 -07:00
mm: split gfp_mask and mapping flags into separate fields
2016-10-11 15:06:34 -07:00
PCI changes for the v4.9 merge window:
2016-10-07 11:46:37 -07:00
locking/percpu-rwsem: Add down_read_preempt_disable()
2016-09-22 15:25:54 +02:00
Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next
2016-10-05 10:11:24 -07:00
net: ethernet: mediatek: add extension of phy-mode for TRGMII
2016-09-22 08:21:21 -04:00
pipe: fix comment in pipe_buf_operations
2016-10-05 18:24:00 -04:00
Merge remote-tracking branch 'jk/vfs' into work.misc
2016-10-08 11:06:08 -04:00
posix_acl: uapi header split
2016-09-27 21:52:00 -04:00
printk: reinstate KERN_CONT for printing continuation lines
2016-10-09 12:23:38 -07:00
nsfs: add ioctl to get a parent namespace
2016-09-22 19:59:41 -05:00
include/linux/property.h: fix typo/compile error
2016-09-30 15:26:52 -07:00
ptp_clock: future-proofing drivers against PTP subsystem becoming optional
2016-09-22 02:18:33 -04:00
radix-tree: 'slot' can be NULL in radix_tree_next_slot()
2016-10-11 15:06:30 -07:00
random: remove unused randomize_range()
2016-10-11 15:06:32 -07:00
relay: Use irq_work instead of plain timer for deferred wakeup
2016-10-11 15:06:32 -07:00
remoteproc: Split driver and consumer dereferencing
2016-10-02 22:50:21 -07:00
thp: reduce usage of huge zero page's atomic counter
2016-10-07 18:46:28 -07:00
ipc/sem.c: fix complex_count vs. simple op race
2016-10-11 15:06:33 -07:00
seq/proc: modify seq_put_decimal_[u]ll to take a const char *, not char
2016-10-07 18:46:30 -07:00
ACPI: parse SPCR and enable matching console
2016-09-28 17:46:46 +02:00
Merge branch 'work.splice_read' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2016-10-07 15:36:58 -07:00
new iov_iter flavour: pipe-backed
2016-10-05 18:23:36 -04:00
mm, swap: add swap_cluster_list
2016-10-07 18:46:27 -07:00
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace
2016-10-06 09:52:23 -07:00
thermal: Enhance thermal_zone_device_update for events
2016-09-27 14:35:21 +08:00
thread_info: Use unsigned long for flags
2016-09-24 09:35:06 +02:00
timekeeping: Include the correct header for errno definitions
2016-09-22 17:02:55 +02:00
u64_stats: Introduce IRQs disabled helpers
2016-09-30 11:46:40 +02:00
Merge branch 'work.splice_read' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
2016-10-10 13:38:49 -07:00
Merge branch 'nsfs-ioctls' into HEAD
2016-09-22 20:00:36 -05:00
sched/wait: Introduce init_wait_entry()
2016-09-30 10:54:03 +02:00
watchdog: add watchdog pretimeout governor framework
2016-10-08 10:27:10 +02:00
mm, vmscan: get rid of throttle_vm_writeout
2016-10-07 18:46:27 -07:00
vfs: Remove {get,set,remove}xattr inode operations
2016-10-07 21:48:36 -04:00
