Chao Yu
e0edeff7d7
f2fs: support data compression ...
This patch tries to support compression in f2fs.
- New term named cluster is defined as basic unit of compression, file can
be divided into multiple clusters logically. One cluster includes 4 << n
(n >= 0) logical pages, compression size is also cluster size, each of
cluster can be compressed or not.
- In cluster metadata layout, one special flag is used to indicate cluster
is compressed one or normal one, for compressed cluster, following metadata
maps cluster to [1, 4 << n - 1] physical blocks, in where f2fs stores
data including compress header and compressed data.
- In order to eliminate write amplification during overwrite, F2FS only
support compression on write-once file, data can be compressed only when
all logical blocks in file are valid and cluster compress ratio is lower
than specified threshold.
- To enable compression on regular inode, there are three ways:
* chattr +c file
* chattr +c dir; touch dir/file
* mount w/ -o compress_extension=ext; touch file.ext
Compress metadata layout:
[Dnode Structure]
+-----------------------------------------------+
| cluster 1 | cluster 2 | ......... | cluster N |
+-----------------------------------------------+
. . . .
. . . .
. Compressed Cluster . . Normal Cluster .
+----------+---------+---------+---------+ +---------+---------+---------+---------+
|compr flag| block 1 | block 2 | block 3 | | block 1 | block 2 | block 3 | block 4 |
+----------+---------+---------+---------+ +---------+---------+---------+---------+
. .
. .
. .
+-------------+-------------+----------+----------------------------+
| data length | data chksum | reserved | compressed data |
+-------------+-------------+----------+----------------------------+
Changelog:
20190326:
- fix error handling of read_end_io().
- remove unneeded comments in f2fs_encrypt_one_page().
20190327:
- fix wrong use of f2fs_cluster_is_full() in f2fs_mpage_readpages().
- don't jump into loop directly to avoid uninitialized variables.
- add TODO tag in error path of f2fs_write_cache_pages().
20190328:
- fix wrong merge condition in f2fs_read_multi_pages().
- check compressed file in f2fs_post_read_required().
20190401
- allow overwrite on non-compressed cluster.
- check cluster meta before writing compressed data.
20190402
- don't preallocate blocks for compressed file.
- add lz4 compress algorithm
- process multiple post read works in one workqueue
Now f2fs supports processing post read work in multiple workqueue,
it shows low performance due to schedule overhead of multiple
workqueue executing orderly.
20190921
- compress: support buffered overwrite
C: compress cluster flag
V: valid block address
N: NEW_ADDR
One cluster contain 4 blocks
before overwrite after overwrite
- VVVV -> CVNN
- CVNN -> VVVV
- CVNN -> CVNN
- CVNN -> CVVV
- CVVV -> CVNN
- CVVV -> CVVV
20191029
- add kconfig F2FS_FS_COMPRESSION to isolate compression related
codes, add kconfig F2FS_FS_{LZO,LZ4} to cover backend algorithm.
note that: will remove lzo backend if Jaegeuk agreed that too.
- update codes according to Eric's comments.
20191101
- apply fixes from Jaegeuk
20191113
- apply fixes from Jaegeuk
- split workqueue for fsverity
20191216
- apply fixes from Jaegeuk
[Jaegeuk Kim]
- add tracepoint for f2fs_{,de}compress_pages()
- fix many bugs and add some compression stats
- fix overwrite/mmap bugs
- address 32bit build error, reported by Geert.
- bug fixes when handling errors and i_compressed_blocks
Reported-by: <noreply@ellerman.id.au>
Signed-off-by: Chao Yu <yuchao0@huawei.com>
Signed-off-by: Jaegeuk Kim <jaegeuk@kernel.org>
2020-01-28 10:35:38 -08:00
can: af_can: export can_sock_destruct()
2019-11-13 10:42:33 +01:00
libceph: add function that clears osd client's abort_err
2019-09-16 12:06:23 +02:00
clk: add include guard to clk-conf.h
2019-09-17 10:27:46 -07:00
net: dsa: sja1105: Fix sleeping while atomic in .port_hwtstamp_set
2019-10-02 12:19:53 -04:00
Char/Misc driver patches for 5.4-rc1
2019-09-18 11:14:31 -07:00
gpiolib: Initialize the hardware with a callback
2019-10-15 01:18:46 +02:00
staging: greybus: fix more header declarations
2019-08-28 17:24:22 +02:00
chrome platform changes for v5.4
2019-09-19 14:14:28 -07:00
Merge branch 'next' into for-linus
2019-09-16 09:56:27 -07:00
mailbox: mediatek: cmdq: clear the event in cmdq initial flow
2019-09-17 00:40:05 -05:00
- New Drivers
2019-09-23 19:37:49 -07:00
net/mlx5: Fix flow counter list auto bits struct
2019-10-29 16:27:17 -07:00
dma-mapping updates for 5.4:
2019-09-19 13:27:23 -07:00
This pull-request contains the following changes for MTD:
2019-09-21 10:59:54 -07:00
netfilter: remove CONFIG_NETFILTER checks from headers.
2019-09-13 12:47:36 +02:00
netfilter: remove CONFIG_NETFILTER checks from headers.
2019-09-13 12:47:36 +02:00
netfilter: remove CONFIG_NETFILTER checks from headers.
2019-09-13 12:47:36 +02:00
netfilter: remove CONFIG_NETFILTER checks from headers.
2019-09-13 12:47:36 +02:00
netfilter: remove CONFIG_NETFILTER checks from headers.
2019-09-13 12:47:36 +02:00
pinctrl/gpio: Take MUX usage into account
2019-08-23 11:09:41 +02:00
dmaengine: imx-sdma: fix size check for sdma script_number
2019-10-14 13:33:44 +05:30
RDMA subsystem updates for 5.4
2019-09-21 10:26:24 -07:00
regulator: add missing 'static inline' to a helper's stub
2019-09-03 12:37:16 +01:00
exit/exec: Seperate mm_release()
2019-11-29 10:10:10 +01:00
drm main pull for 5.4-rc1
2019-09-19 16:24:24 -07:00
soundwire: stream: make stream name a const pointer
2019-09-04 14:57:28 +05:30
SUNRPC: Destroy the back channel when we destroy the host transport
2019-10-30 12:04:35 -04:00
USB changes for 5.4-rc1
2019-09-18 10:33:46 -07:00
Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2019-09-28 08:14:15 -07:00
kill LOOKUP_NO_EVAL, don't bother including namei.h from audit.h
2019-08-30 21:29:32 -04:00
writeback, memcg: Implement foreign dirty flushing
2019-08-27 09:22:38 -06:00
writeback: Separate out wb_get_lookup() from wb_get_create()
2019-08-27 09:22:38 -06:00
backlight: Expose brightness curve type through sysfs
2019-09-02 15:55:03 +01:00
bitmap.h: fix kernel-doc warning and typo
2019-10-14 15:04:01 -07:00
lib: introduce copy_struct_from_user() helper
2019-10-01 15:45:03 +02:00
blkcg: separate blkcg_conf_get_disk() out of blkg_conf_prep()
2019-08-28 21:17:04 -06:00
block: Delay default elevator initialization
2019-09-05 19:52:34 -06:00
block: add zone open, close and finish operations
2020-01-07 16:02:44 -08:00
block: don't handle bio based drivers in blk_revalidate_disk_zones
2020-01-07 16:02:50 -08:00
bpf: Change size to u64 for bpf_map_{area_alloc, charge_init}()
2019-10-31 21:41:33 +01:00
bpf: introduce verifier internal test flag
2019-08-28 00:30:11 +02:00
kprobes: Prohibit probing on BUG() and WARN() address
2019-09-05 10:15:16 +02:00
Merge branches 'clk-init-destroy', 'clk-doc', 'clk-imx' and 'clk-allwinner' into clk-next
2019-09-19 15:31:27 -07:00
We have a small collection of core framework updates this time, mostly around
2019-09-20 15:45:07 -07:00
mm, compaction: raise compaction priority after it withdrawns
2019-09-24 15:54:10 -07:00
futex: Move futex exit handling into futex code
2019-11-29 10:10:08 +01:00
include/linux/compiler.h: fix Oops for Clang-compiled kernels
2019-09-08 14:53:58 +02:00
compiler_attributes.h: Add 'fallthrough' pseudo keyword for switch/case use
2019-10-11 09:26:05 -07:00
Make use of gcc 9's "asm inline()" (Rasmus Villemoes):
2019-09-21 09:47:19 -07:00
driver-core: add include guard to linux/container.h
2019-09-04 13:38:26 +02:00
cpu/speculation: Uninline and export CPU mitigations helpers
2019-11-04 12:22:02 +01:00
cpufreq: Use per-policy frequency QoS
2019-10-21 02:05:21 +02:00
Power management updates for 5.4-rc1
2019-09-17 19:15:14 -07:00
cpuidle-haltpoll: vcpu hotplug support
2019-09-03 09:36:36 +02:00
cpumask: nicer for_each_cpumask_and() signature
2019-09-25 17:51:40 -07:00
selinux/stable-5.4 PR 20190917
2019-09-23 11:21:04 -07:00
PM / devfreq: events: extend events by type of counted data
2019-08-25 12:44:38 +09:00
block: rework zone reporting
2020-01-07 16:02:47 -08:00
USB changes for 5.4-rc1
2019-09-18 10:33:46 -07:00
dma-mapping updates for 5.4:
2019-09-19 13:27:23 -07:00
dma-mapping: remove arch_dma_mmap_pgprot
2019-08-29 16:43:22 +02:00
dynamic_debug: provide dynamic_hex_dump stub
2019-10-22 12:54:44 -07:00
EDAC: Prefer 'unsigned int' to bare use of 'unsigned'
2019-09-03 19:21:19 +02:00
x86, efi: Never relocate kernel below lowest acceptable address
2019-10-31 09:40:19 +01:00
block: Introduce elevator features
2019-09-05 19:52:33 -06:00
symbol namespaces: revert to previous __ksymtab name scheme
2019-10-18 15:32:52 +02:00
f2fs: support data compression
2020-01-28 10:35:38 -08:00
bpf: Fix use after free in subprog's jited symbol removal
2019-10-22 11:26:09 -07:00
Highlights:
2019-09-27 17:00:27 -07:00
fuse update for 5.4
2019-09-25 09:55:59 -07:00
fscrypt: add support for IV_INO_LBLK_64 policies
2020-01-08 15:26:58 -08:00
futex: Add mutex around futex exit
2019-11-29 10:10:13 +01:00
device property: Remove duplicate test for NULL
2019-08-26 11:40:24 +02:00
net: fix sk_page_frag() recursion from memory reclaim
2019-10-28 16:17:31 -07:00
Linux 5.3-rc7
2019-09-05 11:40:54 +02:00
staging: greybus: move core include files to include/linux/greybus/
2019-08-27 19:02:59 +02:00
HID: core: Add printk_once variants to hid_warn() etc
2019-09-18 15:58:13 +02:00
hrtimer: Add kernel doc annotation for HRTIMER_MODE_HARD
2019-08-28 13:01:25 +02:00
libnvdimm fixes v5.4-rc1
2019-09-29 10:33:41 -07:00
mm: introduce page_size()
2019-09-24 15:54:08 -07:00
hwmon: Fix HWMON_P_MIN_ALARM mask
2019-10-02 06:36:50 -07:00
- First round of vmbus hibernation support from Dexuan Cui.
2019-09-24 12:36:31 -07:00
Merge branch 'i2c/for-5.4' of git://git.kernel.org/pub/scm/linux/kernel/git/wsa/linux
2019-09-24 16:48:02 -07:00
powercap: idle_inject: Use higher resolution for idle injection
2019-09-03 11:33:29 +02:00
XArray updates for 5.4
2019-11-08 08:46:49 -08:00
net: remove unnecessary variables and callback
2019-10-24 14:53:49 -07:00
team: fix nested locking lockdep warning
2019-10-24 14:53:48 -07:00
net: remove unnecessary variables and callback
2019-10-24 14:53:49 -07:00
Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2019-09-28 08:14:15 -07:00
posix-cpu-timers: Create a container struct
2019-08-28 11:50:33 +02:00
iommu/vt-d: Fix QI_DEV_IOTLB_PFSID and QI_DEV_EIOTLB_PFSID macros
2019-11-11 16:10:54 +01:00
Power management updates for 5.4-rc1
2019-09-17 19:15:14 -07:00
augmented rbtree: add new RB_DECLARE_CALLBACKS_MAX macro
2019-09-25 17:51:39 -07:00
Merge branch 'arm/smmu' into arm/mediatek
2019-08-30 16:12:10 +02:00
iomap: move the iomap_dio_rw ->end_io callback into a structure
2019-09-19 15:32:45 -07:00
iommu: Add helpers to set/get default domain type
2019-08-23 10:09:58 +02:00
jbd2: Fix possible overflow in jbd2_log_space_left()
2019-12-13 08:42:53 +01:00
kernel.h: Add non_block_start/end()
2019-09-07 04:28:05 -03:00
kernfs: fix ino wrap-around detection
2019-12-13 08:42:53 +01:00
Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2019-09-28 08:14:15 -07:00
afs: Provide an RCU-capable key lookup
2019-09-02 11:43:54 +01:00
kgdb: don't use a notifier to enter kgdb at panic; call directly
2019-09-25 17:51:40 -07:00
khugepaged: enable collapse pmd for pte-mapped THP
2019-09-24 15:54:11 -07:00
Bugfixes: unwinding of KVM_CREATE_VM failure,
2019-11-12 13:19:15 -08:00
leds: core: Fix leds.h structure documentation
2019-10-08 22:05:58 +02:00
libnvdimm/security: Introduce a 'frozen' attribute
2019-08-29 13:49:13 -07:00
Merge branch 'next-lockdown' of git://git.kernel.org/pub/scm/linux/kernel/git/jmorris/linux-security
2019-09-28 08:14:15 -07:00
include: mdio: Add driver data helpers
2019-09-05 12:32:05 +02:00
mm, memcg: make scan aggression always exclude protection
2019-10-07 15:47:20 -07:00
mm/memory_hotplug: fix try_offline_node()
2019-11-15 18:34:00 -08:00
Revert "Revert "mm, thp: restore node-local hugepage allocations""
2019-09-28 14:05:38 -07:00
libnvdimm fixes v5.4-rc1
2019-09-29 10:33:41 -07:00
net: phy: micrel: Update KSZ87xx PHY name
2019-10-17 16:31:52 -07:00
net: phy: fix write to mii-ctrl1000 register
2019-10-04 18:11:08 -07:00
rfkill: allocate static minor
2019-12-13 08:43:18 +01:00
mm: thp: handle page cache THP correctly in PageTransCompoundMap
2019-11-06 08:28:58 -08:00
mm: thp: handle page cache THP correctly in PageTransCompoundMap
2019-11-06 08:28:58 -08:00
fork: improve error message for corrupted page tables
2019-09-25 17:51:40 -07:00
mm/mmu_notifiers: annotate with might_sleep()
2019-09-07 04:28:04 -03:00
mm: thp: extract split_queue_* into a struct
2019-09-24 15:54:11 -07:00
Merge branch 'next-integrity' of git://git.kernel.org/pub/scm/linux/kernel/git/zohar/linux-integrity
2019-09-27 19:37:27 -07:00
fs/namei.c: keep track of nd->root refcount status
2019-09-03 09:30:45 -04:00
net: remove unnecessary variables and callback
2019-10-24 14:53:49 -07:00
netfilter: remove CONFIG_NETFILTER checks from headers.
2019-09-13 12:47:36 +02:00
netfilter: move nf_bridge_frag_data struct definition to a more appropriate header.
2019-09-13 12:35:33 +02:00
netfilter: move nf_bridge_frag_data struct definition to a more appropriate header.
2019-09-13 12:35:33 +02:00
NFS: Refactor nfs_instantiate() for dentry referencing callers
2019-09-20 15:15:24 -04:00
nvme-pci: Add support for variable IO SQ element size
2019-08-29 12:55:02 -07:00
gpio: of: Make of_gpio_simple_xlate() private
2019-09-11 14:46:02 +01:00
padata: remove cpu_index from the parallel_queue
2019-09-13 21:15:41 +10:00
mm: thp: handle page cache THP correctly in PageTransCompoundMap
2019-11-06 08:28:58 -08:00
mm, page_owner: rename flag indicating that page is allocated
2019-10-14 15:04:00 -07:00
mm: page cache: store only head pages in i_pages
2019-09-24 15:54:08 -07:00
pagewalk: separate function pointers from iterator data
2019-09-07 04:28:04 -03:00
pci-v5.4-changes
2019-09-23 19:16:01 -07:00
PCI/ACPI: Move _HPP & _HPX functions to pci-acpi.c
2019-08-28 15:25:53 -05:00
pci-v5.4-changes
2019-09-23 19:16:01 -07:00
perf/headers: Fix spelling s/EACCESS/EACCES/, s/privilidge/privilege/
2019-10-28 11:02:01 +01:00
net: phy: extract pause mode
2019-10-04 18:11:08 -07:00
net: fixed_phy: Add forward declaration for struct gpio_desc;
2019-09-05 11:54:14 +02:00
driver core: platform: Add platform_get_irq_byname_optional()
2019-10-07 12:52:44 +02:00
PM: QoS: Drop frequency QoS types from device PM QoS
2019-10-21 02:05:21 +02:00
posix-cpu-timers: Always clear head pointer on dequeue
2019-09-05 21:16:22 +02:00
lib/hexdump: make print_hex_dump_bytes() a nop on !DEBUG builds
2019-09-25 17:51:39 -07:00
software node: Add software_node_find_by_name()
2019-08-26 11:35:13 +02:00
pwm: Ensure pwm_apply_state() doesn't modify the state argument
2019-09-21 03:25:10 +02:00
quota: fix wrong condition in is_quota_modification()
2019-09-12 12:09:16 +02:00
radix tree: Remove radix_tree_iter_find
2019-11-01 22:26:34 -04:00
vfs: Convert ramfs, shmem, tmpfs, devtmpfs, rootfs to use the new mount API
2019-09-12 21:05:34 -04:00
fdt: add support for rng-seed
2019-08-23 16:39:26 +01:00
augmented rbtree: rework the RB_DECLARE_CALLBACKS macro definition
2019-09-25 17:51:39 -07:00
Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-09-16 17:25:49 -07:00
rcu: Don't include <linux/ktime.h> in rcutiny.h
2019-08-26 16:27:01 -07:00
tasks, sched/core: With a grace period after finish_task_switch(), remove unnecessary code
2019-09-25 17:42:29 +02:00
reset: fix reset_control_ops kerneldoc comment
2019-10-28 12:18:23 +01:00
reset: fix reset_control_get_exclusive kerneldoc comment
2019-10-24 10:26:33 +02:00
cifs: Add support for root file systems
2019-09-16 11:43:38 -05:00
futex: Add mutex around futex exit
2019-11-29 10:10:13 +01:00
efi/efi_test: Lock down /dev/efi_test and require CAP_SYS_ADMIN
2019-10-31 09:40:21 +01:00
vfs: Convert ramfs, shmem, tmpfs, devtmpfs, rootfs to use the new mount API
2019-09-12 21:05:34 -04:00
mm: shrinker: make shrinker not depend on memcg kmem
2019-09-24 15:54:11 -07:00
udp: drop skb extensions before marking skb stateless
2019-11-22 09:28:46 -08:00
net: skmsg: fix TLS 1.3 crash with full sk_msg
2019-12-04 22:31:01 +01:00
mm, sl[aou]b: guarantee natural alignment for kmalloc(power-of-two)
2019-10-07 15:47:20 -07:00
net: increase SOMAXCONN to 4096
2019-10-31 14:01:40 -07:00
statx: define STATX_ATTR_VERITY
2020-01-08 15:26:59 -08:00
net: stmmac: implement support for passive mode converters via dt
2019-09-11 15:27:09 +01:00
lib/string: Make memzero_explicit() inline instead of external
2019-10-08 13:27:05 +02:00
mm: introduce MADV_PAGEOUT
2019-09-25 17:51:41 -07:00
swiotlb: Split size parameter to map/unmap APIs
2019-09-11 12:34:29 +02:00
ipc: fix sparc64 ipc() wrapper
2019-09-07 21:42:25 +02:00
sysfs: Fixes __BIN_ATTR_WO() macro
2019-10-02 10:06:45 +02:00
block: centralize PI remapping logic to the block layer
2019-09-17 20:03:49 -06:00
tcp: add rcu protection around tp->fastopen_rsk
2019-10-13 10:13:08 -07:00
uaccess: add missing __must_check attributes
2019-09-25 17:51:40 -07:00
thunderbolt: Add support for Intel Ice Lake
2019-08-26 12:15:06 +03:00
vfs: Add file timestamp range support
2019-08-30 07:27:17 -07:00
timekeeping/vsyscall: Prevent math overflow in BOOTTIME update
2019-08-23 02:12:11 +02:00
posix-cpu-timers: Utilize timerqueue for storage
2019-08-28 11:50:43 +02:00
Merge branch 'sched-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-09-16 17:25:49 -07:00
efi/tpm: Only set 'efi_tpm_final_log_size' after successful event log parsing
2019-10-07 15:24:36 +02:00
tracing: Make exported ftrace_set_clr_event non-static
2019-08-31 06:51:49 -04:00
uaccess: implement a proper unsafe_copy_to_user() and switch filldir over to it
2019-10-07 12:56:48 -07:00
vsock/virtio: remove unused 'work' field from 'struct virtio_vsock_pkt'
2019-10-28 04:25:04 -04:00
mm/vmalloc: modify struct vmap_area to reduce its size
2019-09-24 15:54:10 -07:00
w1: add 1-wire master driver for IP block found in SGI ASICs
2019-09-04 14:34:31 +02:00
Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
2019-09-17 12:35:15 -07:00
workqueue: unconfine alloc/apply/free_workqueue_attrs()
2019-09-13 21:15:39 +10:00
writeback, memcg: Implement cgroup_writeback_by_id()
2019-08-27 09:22:38 -06:00
xarray.h: fix kernel-doc warning
2019-10-14 15:04:01 -07:00
zpool: add malloc_support_movable to zpool_driver
2019-09-24 15:54:12 -07:00
