Chao Yu
e0edeff7d7
f2fs: support data compression ...
This patch tries to support compression in f2fs.
- New term named cluster is defined as basic unit of compression, file can
be divided into multiple clusters logically. One cluster includes 4 << n
(n >= 0) logical pages, compression size is also cluster size, each of
cluster can be compressed or not.
- In cluster metadata layout, one special flag is used to indicate cluster
is compressed one or normal one, for compressed cluster, following metadata
maps cluster to [1, 4 << n - 1] physical blocks, in where f2fs stores
data including compress header and compressed data.
- In order to eliminate write amplification during overwrite, F2FS only
support compression on write-once file, data can be compressed only when
all logical blocks in file are valid and cluster compress ratio is lower
than specified threshold.
- To enable compression on regular inode, there are three ways:
* chattr +c file
* chattr +c dir; touch dir/file
* mount w/ -o compress_extension=ext; touch file.ext
Compress metadata layout:
[Dnode Structure]
+-----------------------------------------------+
| cluster 1 | cluster 2 | ......... | cluster N |
+-----------------------------------------------+
. . . .
. . . .
. Compressed Cluster . . Normal Cluster .
+----------+---------+---------+---------+ +---------+---------+---------+---------+
|compr flag| block 1 | block 2 | block 3 | | block 1 | block 2 | block 3 | block 4 |
+----------+---------+---------+---------+ +---------+---------+---------+---------+
. .
. .
. .
+-------------+-------------+----------+----------------------------+
| data length | data chksum | reserved | compressed data |
+-------------+-------------+----------+----------------------------+
Changelog:
20190326:
- fix error handling of read_end_io().
- remove unneeded comments in f2fs_encrypt_one_page().
20190327:
- fix wrong use of f2fs_cluster_is_full() in f2fs_mpage_readpages().
- don't jump into loop directly to avoid uninitialized variables.
- add TODO tag in error path of f2fs_write_cache_pages().
20190328:
- fix wrong merge condition in f2fs_read_multi_pages().
- check compressed file in f2fs_post_read_required().
20190401
- allow overwrite on non-compressed cluster.
- check cluster meta before writing compressed data.
20190402
- don't preallocate blocks for compressed file.
- add lz4 compress algorithm
- process multiple post read works in one workqueue
Now f2fs supports processing post read work in multiple workqueue,
it shows low performance due to schedule overhead of multiple
workqueue executing orderly.
20190921
- compress: support buffered overwrite
C: compress cluster flag
V: valid block address
N: NEW_ADDR
One cluster contain 4 blocks
before overwrite after overwrite
- VVVV -> CVNN
- CVNN -> VVVV
- CVNN -> CVNN
- CVNN -> CVVV
- CVVV -> CVNN
- CVVV -> CVVV
20191029
- add kconfig F2FS_FS_COMPRESSION to isolate compression related
codes, add kconfig F2FS_FS_{LZO,LZ4} to cover backend algorithm.
note that: will remove lzo backend if Jaegeuk agreed that too.
- update codes according to Eric's comments.
20191101
- apply fixes from Jaegeuk
20191113
- apply fixes from Jaegeuk
- split workqueue for fsverity
20191216
- apply fixes from Jaegeuk
[Jaegeuk Kim]
- add tracepoint for f2fs_{,de}compress_pages()
- fix many bugs and add some compression stats
- fix overwrite/mmap bugs
- address 32bit build error, reported by Geert.
- bug fixes when handling errors and i_compressed_blocks
Reported-by: <noreply@ellerman.id.au>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-01-28 10:35:38 -08:00
can: af_can: export can_sock_destruct()
2019-11-13 10:42:33 +01:00
net: dsa: sja1105: Fix sleeping while atomic in .port_hwtstamp_set
2019-10-02 12:19:53 -04:00
gpiolib: Initialize the hardware with a callback
2019-10-15 01:18:46 +02:00
- New Drivers
2019-09-23 19:37:49 -07:00
net/mlx5: Fix flow counter list auto bits struct
2019-10-29 16:27:17 -07:00
dmaengine: imx-sdma: fix size check for sdma script_number
2019-10-14 13:33:44 +05:30
exit/exec: Seperate mm_release()
2019-11-29 10:10:10 +01:00
SUNRPC: Destroy the back channel when we destroy the host transport
2019-10-30 12:04:35 -04:00
Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2019-09-28 08:14:15 -07:00
bitmap.h: fix kernel-doc warning and typo
2019-10-14 15:04:01 -07:00
lib: introduce copy_struct_from_user() helper
2019-10-01 15:45:03 +02:00
block: add zone open, close and finish operations
2020-01-07 16:02:44 -08:00
block: don't handle bio based drivers in blk_revalidate_disk_zones
2020-01-07 16:02:50 -08:00
bpf: Change size to u64 for bpf_map_{area_alloc, charge_init}()
2019-10-31 21:41:33 +01:00
mm, compaction: raise compaction priority after it withdrawns
2019-09-24 15:54:10 -07:00
futex: Move futex exit handling into futex code
2019-11-29 10:10:08 +01:00
compiler_attributes.h: Add 'fallthrough' pseudo keyword for switch/case use
2019-10-11 09:26:05 -07:00
cpu/speculation: Uninline and export CPU mitigations helpers
2019-11-04 12:22:02 +01:00
cpufreq: Use per-policy frequency QoS
2019-10-21 02:05:21 +02:00
cpumask: nicer for_each_cpumask_and() signature
2019-09-25 17:51:40 -07:00
selinux/stable-5.4 PR 20190917
2019-09-23 11:21:04 -07:00
block: rework zone reporting
2020-01-07 16:02:47 -08:00
dynamic_debug: provide dynamic_hex_dump stub
2019-10-22 12:54:44 -07:00
x86, efi: Never relocate kernel below lowest acceptable address
2019-10-31 09:40:19 +01:00
symbol namespaces: revert to previous __ksymtab name scheme
2019-10-18 15:32:52 +02:00
f2fs: support data compression
2020-01-28 10:35:38 -08:00
bpf: Fix use after free in subprog's jited symbol removal
2019-10-22 11:26:09 -07:00
Highlights:
2019-09-27 17:00:27 -07:00
fuse update for 5.4
2019-09-25 09:55:59 -07:00
fscrypt: add support for IV_INO_LBLK_64 policies
2020-01-08 15:26:58 -08:00
futex: Add mutex around futex exit
2019-11-29 10:10:13 +01:00
net: fix sk_page_frag() recursion from memory reclaim
2019-10-28 16:17:31 -07:00
libnvdimm fixes v5.4-rc1
2019-09-29 10:33:41 -07:00
mm: introduce page_size()
2019-09-24 15:54:08 -07:00
hwmon: Fix HWMON_P_MIN_ALARM mask
2019-10-02 06:36:50 -07:00
- First round of vmbus hibernation support from Dexuan Cui.
2019-09-24 12:36:31 -07:00
Merge branch 'i2c/for-5.4' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux
2019-09-24 16:48:02 -07:00
XArray updates for 5.4
2019-11-08 08:46:49 -08:00
net: remove unnecessary variables and callback
2019-10-24 14:53:49 -07:00
team: fix nested locking lockdep warning
2019-10-24 14:53:48 -07:00
net: remove unnecessary variables and callback
2019-10-24 14:53:49 -07:00
Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2019-09-28 08:14:15 -07:00
iommu/vt-d: Fix QI_DEV_IOTLB_PFSID and QI_DEV_EIOTLB_PFSID macros
2019-11-11 16:10:54 +01:00
augmented rbtree: add new RB_DECLARE_CALLBACKS_MAX macro
2019-09-25 17:51:39 -07:00
jbd2: Fix possible overflow in jbd2_log_space_left()
2019-12-13 08:42:53 +01:00
kernfs: fix ino wrap-around detection
2019-12-13 08:42:53 +01:00
Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2019-09-28 08:14:15 -07:00
kgdb: don't use a notifier to enter kgdb at panic; call directly
2019-09-25 17:51:40 -07:00
khugepaged: enable collapse pmd for pte-mapped THP
2019-09-24 15:54:11 -07:00
Bugfixes: unwinding of KVM_CREATE_VM failure,
2019-11-12 13:19:15 -08:00
leds: core: Fix leds.h structure documentation
2019-10-08 22:05:58 +02:00
Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2019-09-28 08:14:15 -07:00
mm, memcg: make scan aggression always exclude protection
2019-10-07 15:47:20 -07:00
mm/memory_hotplug: fix try_offline_node()
2019-11-15 18:34:00 -08:00
Revert "Revert "mm, thp: restore node-local hugepage allocations""
2019-09-28 14:05:38 -07:00
libnvdimm fixes v5.4-rc1
2019-09-29 10:33:41 -07:00
net: phy: micrel: Update KSZ87xx PHY name
2019-10-17 16:31:52 -07:00
net: phy: fix write to mii-ctrl1000 register
2019-10-04 18:11:08 -07:00
rfkill: allocate static minor
2019-12-13 08:43:18 +01:00
mm: thp: handle page cache THP correctly in PageTransCompoundMap
2019-11-06 08:28:58 -08:00
mm: thp: handle page cache THP correctly in PageTransCompoundMap
2019-11-06 08:28:58 -08:00
fork: improve error message for corrupted page tables
2019-09-25 17:51:40 -07:00
mm: thp: extract split_queue_* into a struct
2019-09-24 15:54:11 -07:00
Merge branch 'next-integrity' of git://git.kernel.org/pub/scm/linux/kernel/git/zohar/linux-integrity
2019-09-27 19:37:27 -07:00
net: remove unnecessary variables and callback
2019-10-24 14:53:49 -07:00
mm: thp: handle page cache THP correctly in PageTransCompoundMap
2019-11-06 08:28:58 -08:00
mm, page_owner: rename flag indicating that page is allocated
2019-10-14 15:04:00 -07:00
mm: page cache: store only head pages in i_pages
2019-09-24 15:54:08 -07:00
pci-v5.4-changes
2019-09-23 19:16:01 -07:00
pci-v5.4-changes
2019-09-23 19:16:01 -07:00
perf/headers: Fix spelling s/EACCESS/EACCES/, s/privilidge/privilege/
2019-10-28 11:02:01 +01:00
net: phy: extract pause mode
2019-10-04 18:11:08 -07:00
driver core: platform: Add platform_get_irq_byname_optional()
2019-10-07 12:52:44 +02:00
PM: QoS: Drop frequency QoS types from device PM QoS
2019-10-21 02:05:21 +02:00
lib/hexdump: make print_hex_dump_bytes() a nop on !DEBUG builds
2019-09-25 17:51:39 -07:00
radix tree: Remove radix_tree_iter_find
2019-11-01 22:26:34 -04:00
augmented rbtree: rework the RB_DECLARE_CALLBACKS macro definition
2019-09-25 17:51:39 -07:00
tasks, sched/core: With a grace period after finish_task_switch(), remove unnecessary code
2019-09-25 17:42:29 +02:00
reset: fix reset_control_ops kerneldoc comment
2019-10-28 12:18:23 +01:00
reset: fix reset_control_get_exclusive kerneldoc comment
2019-10-24 10:26:33 +02:00
futex: Add mutex around futex exit
2019-11-29 10:10:13 +01:00
efi/efi_test: Lock down /dev/efi_test and require CAP_SYS_ADMIN
2019-10-31 09:40:21 +01:00
mm: shrinker: make shrinker not depend on memcg kmem
2019-09-24 15:54:11 -07:00
udp: drop skb extensions before marking skb stateless
2019-11-22 09:28:46 -08:00
net: skmsg: fix TLS 1.3 crash with full sk_msg
2019-12-04 22:31:01 +01:00
mm, sl[aou]b: guarantee natural alignment for kmalloc(power-of-two)
2019-10-07 15:47:20 -07:00
net: increase SOMAXCONN to 4096
2019-10-31 14:01:40 -07:00
statx: define STATX_ATTR_VERITY
2020-01-08 15:26:59 -08:00
lib/string: Make memzero_explicit() inline instead of external
2019-10-08 13:27:05 +02:00
mm: introduce MADV_PAGEOUT
2019-09-25 17:51:41 -07:00
sysfs: Fixes __BIN_ATTR_WO() macro
2019-10-02 10:06:45 +02:00
tcp: add rcu protection around tp->fastopen_rsk
2019-10-13 10:13:08 -07:00
uaccess: add missing __must_check attributes
2019-09-25 17:51:40 -07:00
efi/tpm: Only set 'efi_tpm_final_log_size' after successful event log parsing
2019-10-07 15:24:36 +02:00
uaccess: implement a proper unsafe_copy_to_user() and switch filldir over to it
2019-10-07 12:56:48 -07:00
vsock/virtio: remove unused 'work' field from 'struct virtio_vsock_pkt'
2019-10-28 04:25:04 -04:00
mm/vmalloc: modify struct vmap_area to reduce its size
2019-09-24 15:54:10 -07:00
xarray.h: fix kernel-doc warning
2019-10-14 15:04:01 -07:00
zpool: add malloc_support_movable to zpool_driver
2019-09-24 15:54:12 -07:00
