No description
  • C 98.2%
  • Assembly 1%
  • Makefile 0.3%
  • Shell 0.2%
  • Python 0.1%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Muchun Song 383455a9aa block: fix ordering between checking BLK_MQ_S_STOPPED request adding
commit 96a9fe64bfd486ebeeacf1e6011801ffe89dae18 upstream.

Supposing first scenario with a virtio_blk driver.

CPU0                        CPU1

blk_mq_try_issue_directly()
  __blk_mq_issue_directly()
    q->mq_ops->queue_rq()
      virtio_queue_rq()
        blk_mq_stop_hw_queue()
                            virtblk_done()
  blk_mq_request_bypass_insert()  1) store
                              blk_mq_start_stopped_hw_queue()
                                clear_bit(BLK_MQ_S_STOPPED)       3) store
                                blk_mq_run_hw_queue()
                                  if (!blk_mq_hctx_has_pending()) 4) load
                                    return
                                  blk_mq_sched_dispatch_requests()
  blk_mq_run_hw_queue()
    if (!blk_mq_hctx_has_pending())
      return
    blk_mq_sched_dispatch_requests()
      if (blk_mq_hctx_stopped())  2) load
        return
      __blk_mq_sched_dispatch_requests()

Supposing another scenario.

CPU0                        CPU1

blk_mq_requeue_work()
  blk_mq_insert_request() 1) store
                            virtblk_done()
                              blk_mq_start_stopped_hw_queue()
  blk_mq_run_hw_queues()        clear_bit(BLK_MQ_S_STOPPED)       3) store
                                blk_mq_run_hw_queue()
                                  if (!blk_mq_hctx_has_pending()) 4) load
                                    return
                                  blk_mq_sched_dispatch_requests()
    if (blk_mq_hctx_stopped())  2) load
      continue
    blk_mq_run_hw_queue()

Both scenarios are similar, the full memory barrier should be inserted
between 1) and 2), as well as between 3) and 4) to make sure that either
CPU0 sees BLK_MQ_S_STOPPED is cleared or CPU1 sees dispatch list.
Otherwise, either CPU will not rerun the hardware queue causing
starvation of the request.

The easy way to fix it is to add the essential full memory barrier into
helper of blk_mq_hctx_stopped(). In order to not affect the fast path
(hardware queue is not stopped most of the time), we only insert the
barrier into the slow path. Actually, only slow path needs to care about
missing of dispatching the request to the low-level device driver.

Fixes: 320ae51fee ("blk-mq: new multi-queue block IO queueing mechanism")
Cc: stable@vger.kernel.org
Cc: Muchun Song <muchun.song@linux.dev>
Signed-off-by: Muchun Song <songmuchun@bytedance.com>
Reviewed-by: Ming Lei <ming.lei@redhat.com>
Link: https://lore.kernel.org/r/20241014092934.53630-4-songmuchun@bytedance.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2024-12-14 19:44:38 +01:00
arch arm64: tls: Fix context-switching of tpidrro_el0 when kpti is enabled 2024-12-14 19:44:38 +01:00
block block: fix ordering between checking BLK_MQ_S_STOPPED request adding 2024-12-14 19:44:38 +01:00
certs
crypto crypto: pcrypt - Call crypto layer directly when padata_do_parallel() return -EBUSY 2024-12-14 19:44:23 +01:00
Documentation dt-bindings: clock: axi-clkgen: include AXI clk 2024-12-14 19:44:30 +01:00
drivers serial: 8250: omap: Move pm_runtime_get_sync 2024-12-14 19:44:37 +01:00
fs jfs: xattr: check invalid xattr size more strictly 2024-12-14 19:44:35 +01:00
include cgroup: Move rcu_head up near the top of cgroup_root 2024-12-14 19:44:35 +01:00
init initramfs: avoid filename buffer overrun 2024-12-14 19:44:21 +01:00
ipc
kernel cgroup: Make operations on the cgroup root_list RCU safe 2024-12-14 19:44:35 +01:00
lib
LICENSES
mm mm: revert "mm: shmem: fix data-race in shmem_getattr()" 2024-12-14 19:44:19 +01:00
net netfilter: ipset: add missing range check in bitmap_ip_uadt 2024-12-14 19:44:37 +01:00
samples bpf: Fix the xdp_adjust_tail sample prog issue 2024-12-14 19:44:26 +01:00
scripts
security apparmor: fix 'Do simple duplicate message elimination' 2024-12-14 19:44:34 +01:00
sound ASoC: codecs: Fix atomicity violation in snd_soc_component_get_drvdata() 2024-12-14 19:44:36 +01:00
tools perf trace: Avoid garbage when not printing a syscall's arguments 2024-12-14 19:44:32 +01:00
usr
virt
.clang-format
.cocciconfig
.get_maintainer.ignore
.gitattributes
.gitignore
.mailmap
COPYING
CREDITS
Kbuild
Kconfig
MAINTAINERS
Makefile
README

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.