Commit graph

889,574 commits

Author SHA1 Message Date
qctecmdr
4cb9b09799 Merge "coresight: Forbid enabling sink while reading sink" 2020-06-11 21:35:29 -07:00
qctecmdr
016b45df4e Merge "mm/madvise: pass task and mm to do_madvise" 2020-06-11 21:35:29 -07:00
qctecmdr
dc630fdbf7 Merge "defconfig: msm: Add config fragment for DEBUG_FS disablement" 2020-06-11 21:35:29 -07:00
qctecmdr
0af5ddc604 Merge "drivers: soc: qcom: update sct table for LLCC on lahaina" 2020-06-11 21:35:28 -07:00
qctecmdr
1b78b533e6 Merge "usb: dwc3: debugfs: Add support for 64bit division on 32bit chipsets" 2020-06-11 21:35:28 -07:00
qctecmdr
a77137ef57 Merge "iommu/arm-smmu: add qcom smmuv2 support" 2020-06-11 21:35:28 -07:00
Ritesh Harjani
2144280eab UPSTREAM: mmc: sdhci-msm: Add CQHCI support for sdhci-msm
This adds CQHCI support for sdhci-msm platforms.

Change-Id: I3b4bed0bbd542cc6f2ce461f8c505b2d1c0014a8
Signed-off-by: Ritesh Harjani <riteshh@codeaurora.org>
Signed-off-by: Veerabhadrarao Badiganti <vbadigan@codeaurora.org>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Link: https://lore.kernel.org/r/1579194192-7942-3-git-send-email-vbadigan@codeaurora.org
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Sarthak Garg <sartgarg@qti.qualcomm.com>
Signed-off-by: Veerabhadrarao Badiganti <vbadigan@codeaurora.org>
2020-06-12 09:36:52 +05:30
Veerabhadrarao Badiganti
8d881c1dcc mmc: sdhci: Let a vendor driver supply and update ADMA descriptor size
Let a vendor driver supply the maximum descriptor size that it
can operate on. ADMA descriptor table would be allocated using this
supplied size.
If any SD Host controller is of version prior to v4.10 spec
but supports 16byte descriptor, this change allows them to supply
correct descriptor size for ADMA table allocation.

Also let a vendor driver update the descriptor size by overriding
sdhc_host->desc_size if it has to operates on a different descriptor
sizes in different conditions.

Change-Id: I3f888c1e6a37132a311a8eaee155333d62e7057f
Suggested-by: Adrian Hunter <adrian.hunter@intel.com>
Signed-off-by: Veerabhadrarao Badiganti <vbadigan@codeaurora.org>
Acked-by: Adrian Hunter <adrian.hunter@intel.com>
Link: https://lore.kernel.org/r/1579531122-28341-1-git-send-email-vbadigan@codeaurora.org
Signed-off-by: Ulf Hansson <ulf.hansson@linaro.org>
Signed-off-by: Sarthak Garg <sartgarg@qti.qualcomm.com>
2020-06-12 09:36:08 +05:30
Tao,Zhang
6fef559dc9 coresight: byte-cnter: Replace dev_err with ratelimited API
Replace dev_err with ratelimited API dev_err_ratelimited to avoid
watchdog bark.

Change-Id: I4861bc7d55a39d77020d08bd958ba8fc1311d551
Signed-off-by: Tao,Zhang <taozha@codeaurora.org>
Signed-off-by: Tingwei Zhang <tingwei@codeaurora.org>
2020-06-12 09:42:51 +08:00
Isaac J. Manjarres
66db1f915e arm64: defconfig: Remove unnecessary configs in genericarmv8-64
Certain configs that are enabled in genericarmv8-64 are not
required for the usecases that the kernel is supposed to cater.
For example, IOMMU support is not required for genericarmv8-64,
yet, the drivers for it are built into the kernel image. This
leads to a bloated kernel image with a larger memory footprint,
as well as larger runtime memory footprint, as certain drivers
may allocate data structures that go by unused. Remove certain
extraneous configs.

Change-Id: Iefe8dd2a35a4a29ed97338ec5fea5c147da14eda
Signed-off-by: Isaac J. Manjarres <isaacm@codeaurora.org>
2020-06-11 18:17:50 -07:00
Isaac J. Manjarres
0c5e3da8a2 arm64: defconfig: Disable tracing on genericarmv8-64
Ftracing has been seen to take up a lot of memory (~16 MB).
On certain environments where the amount of memory is scarce,
this is not ideal as it can contribute to memory pressure,
so disable it for genericarmv8-64 for the time being.

Change-Id: I9a8cc66e70caadc53a5cf63a39842c9548fddf25
Signed-off-by: Isaac J. Manjarres <isaacm@codeaurora.org>
2020-06-11 18:16:51 -07:00
Sami Tolvanen
f0ac8d8d72 ANDROID: kbuild: ensure __cfi_check is correctly aligned
On modules with no executable code, LLVM generates a __cfi_check stub,
but won't align it to page size as expected. This change ensures the
function is at the beginning of the .text section and correctly aligned
for the CFI shadow.

Bug: 148458318
Change-Id: I85ea31fa851bc23988f649b021b3ac7e9d9dcb38
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
Git-commit: 4274636cbe
Git-repo: https://android.googlesource.com/kernel/common/
Signed-off-by: Prasad Sodagudi <psodagud@codeaurora.org>
2020-06-11 17:57:31 -07:00
Sami Tolvanen
f31cfc8907 ANDROID: kbuild: merge more sections with LTO
Merge the following new data sections generated by current
LLVM versions:

  .bss..L* .bss..compoundliteral*
  .data..L* .data..compoundliteral*
  .rodata..L* .rodata..compoundliteral*

Also drop the unnecessary rules to merge .rela sections.

Bug: 158605670
Change-Id: I098212bc74dfb6b54d4fd60d90957fea29122a6c
Signed-off-by: Sami Tolvanen <samitolvanen@google.com>
2020-06-11 23:21:36 +00:00
qctecmdr
dfc8288102 Merge "scsi: ufs: Enable WriteBooster" 2020-06-11 10:59:15 -07:00
qctecmdr
e56575eaad Merge "coresight: byte-cnter: Add ETR status check in bypass notifier" 2020-06-11 10:59:14 -07:00
qctecmdr
7c93753e20 Merge "smcinvoke: Move error message to debug" 2020-06-11 10:59:14 -07:00
qctecmdr
dedd7a1e0b Merge "devfreq: memlat: fix potential deadlock during CPU hotplug" 2020-06-11 10:59:14 -07:00
qctecmdr
7aab4d7c7d Merge "PM / devfreq: memlat: increase ratio_ceil limit" 2020-06-11 10:59:14 -07:00
qctecmdr
5963622ada Merge "usb: xhci: Don't check for USB_STATE_NOTATTACHED in sec_event_ring_cleanup" 2020-06-11 10:59:13 -07:00
qctecmdr
33371193a7 Merge "usb: typec: ucsi: Modify IPC log string" 2020-06-11 10:59:13 -07:00
qctecmdr
1b5b9c4b20 Merge "sound: usb: Increase upper bound check for bInterval to 16" 2020-06-11 10:59:13 -07:00
qctecmdr
6a7962bf50 Merge "msm: ipa: Add support to notify wan upstream iface info" 2020-06-11 10:59:13 -07:00
qctecmdr
086b1556d5 Merge "ANDROID: scs: fix recursive spinlock in scs_check_usage" 2020-06-11 10:59:12 -07:00
qctecmdr
c2d54934ff Merge "dwc3-msm: Add support to vote USB FORCE_MEM_CORE_ON" 2020-06-11 10:59:12 -07:00
qctecmdr
8958166013 Merge "soc: qcom: mem-buf: Fix error handling when releasing memory" 2020-06-11 10:59:12 -07:00
Nitheesh Muthuraj
59d94c3a20 msm: kgsl: PERFCNTR_CTRL needs to be at end of restore list
PERFCNTR_CTRL needs to be at the end of restore list to ensure
perfcounters count correctly across IFPC.

Change-Id: Ic0718a28074bf15910e2f6e97c16eb90e69ef917
Signed-off-by: Nitheesh Muthuraj <nmuthuraj@codeaurora.org>
2020-06-11 10:49:26 -07:00
Hareesh Gundu
d39ef48f2b msm: kgsl: Add support to toggle GPU GX CPR loop enable
Add support to toggle GPU GX CPR loop enable to recover
from the GX CPR stall issue.

Change-Id: Ic9f303e213ee8164fa1f97d946c396a4f59301d3
Signed-off-by: Hareesh Gundu <hareeshg@codeaurora.org>
2020-06-11 22:32:42 +05:30
Saravana Kannan
358df78b82 UPSTREAM: driver core: Remove unnecessary is_fwnode_dev variable in device_add()
That variable is no longer necessary. Remove it and also fix a minor
typo in comments.

Signed-off-by: Saravana Kannan <saravanak@google.com>
Link: https://lore.kernel.org/r/20200520034824.79049-2-saravanak@google.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit 2cd38fd15e4ebcfe917a443734820269f8b5ba2b)
Bug: 157691602
Signed-off-by: Saravana Kannan <saravanak@google.com>
Change-Id: Ia7a0122d606211edc404a2733d35182a750daa92
2020-06-11 10:00:07 -07:00
Saravana Kannan
373cfc0f8d UPSTREAM: driver core: Remove check in driver_deferred_probe_force_trigger()
The whole point behind adding driver_deferred_probe_force_trigger() in
commit 716a7a259690 ("driver core: fw_devlink: Add support for batching
fwnode parsing") was to skip the check for driver_deferred_probe_enable.
Otherwise, it's identical to driver_deferred_probe_trigger().

Delete the check in driver_deferred_probe_force_trigger() so that
fw_devlink_pause() and fw_devlink_resume() can kick off deferred probe
as intended. Without doing this forced deferred probe trigger, some
platforms seem to be crashing during boot because they assume probe
order of devices.

Fixes: 716a7a259690 ("driver core: fw_devlink: Add support for batching fwnode parsing")
Signed-off-by: Saravana Kannan <saravanak@google.com>
Link: https://lore.kernel.org/r/20200517173453.157703-1-saravanak@google.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit fefcfc968723caf93318613a08e1f3ad07a6154f)
Bug: 157691602
Signed-off-by: Saravana Kannan <saravanak@google.com>
Change-Id: I2c1378ec24c0b5f257d0c9bbac066ad156dcbafb
2020-06-11 09:59:53 -07:00
Saravana Kannan
b82a5ead71 UPSTREAM: of: platform: Batch fwnode parsing when adding all top level devices
The fw_devlink_pause() and fw_devlink_resume() APIs allow batching the
parsing of the device tree nodes when a lot of devices are added. This
will significantly cut down parsing time (as much a 1 second on some
systems). So, use them when adding devices for all the top level device
tree nodes in a system.

Signed-off-by: Saravana Kannan <saravanak@google.com>
Link: https://lore.kernel.org/r/20200515053500.215929-5-saravanak@google.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit 93d2e4322aa74c1ad1e8c2160608eb9a960d69ff)
Bug: 157691602
Signed-off-by: Saravana Kannan <saravanak@google.com>
Change-Id: I6cc5b4eff7f85b0eb3b6c1c4d81418544a17f532
2020-06-11 09:59:43 -07:00
Saravana Kannan
9acfa80c3d BACKPORT: driver core: fw_devlink: Add support for batching fwnode parsing
The amount of time spent parsing fwnodes of devices can become really
high if the devices are added in an non-ideal order. Worst case can be
O(N^2) when N devices are added. But this can be optimized to O(N) by
adding all the devices and then parsing all their fwnodes in one batch.

This commit adds fw_devlink_pause() and fw_devlink_resume() to allow
doing this.

Signed-off-by: Saravana Kannan <saravanak@google.com>
Link: https://lore.kernel.org/r/20200515053500.215929-4-saravanak@google.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit 716a7a25969003d82ab738179c3f1068a120ed11)
[fixed conflicts due to fw_devlink and fw_devlink permissive mode]
Bug: 157691602
Signed-off-by: Saravana Kannan <saravanak@google.com>
Change-Id: Ibcc32eee577c9325054786a73193d615d54a8f07
2020-06-11 09:57:35 -07:00
Saravana Kannan
bf50baa3a3 BACKPORT: driver core: Look for waiting consumers only for a fwnode's primary device
Commit 4dbe191c046e ("driver core: Add device links from fwnode only for
the primary device") skipped linking a fwnode's secondary device to
the suppliers listed in its fwnode.

However, a fwnode's secondary device can't be found using
get_dev_from_fwnode(). So, there's no point in trying to see if devices
waiting for suppliers might want to link to a fwnode's secondary device.

This commit removes that unnecessary step for devices that aren't a
fwnode's primary device and also moves the code to a more appropriate
part of the file.

Signed-off-by: Saravana Kannan <saravanak@google.com>
Link: https://lore.kernel.org/r/20200515053500.215929-3-saravanak@google.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
(cherry picked from commit 5f5377eaddfc24e5d7562e588d0ff84f9264d7c1)
[conflict fixes due to of_devlink rename to fw_devlink]
Bug: 157691602
Signed-off-by: Saravana Kannan <saravanak@google.com>
Change-Id: Id0455f6bc05d89b0f81d1574a242be6b8ff67975
2020-06-11 09:57:35 -07:00
Can Guo
c2825345ea scsi: ufs-qcom: Remove unnecessary devm_kfree
Resources allocated with devm_* are managed by device driver framework,
meaning the resources will be automatically released when the associated
device is removed. Besides, having a devm_kfree here would cause a
use-after-free scenario: after hba->host is freed up, data inside hba->host
shall still be accessed when device driver framework frees up all resources
associated with hba->dev. hba->host->rcdev, for example, shall be used by
func devm_reset_controller_release() when resources associated with
hba->dev are freed up.

Change-Id: I90bab41c34450a1c814d863b9f3653228f5eeb95
Signed-off-by: Can Guo <cang@codeaurora.org>
2020-06-11 19:38:06 +08:00
Min Guo
2b5f94b925 UPSTREAM: usb: musb: Add support for MediaTek musb controller
This adds support for MediaTek musb controller in
host, peripheral and otg mode.
There are some quirk of MediaTek musb controller, such as:
 -W1C interrupt status registers
 -Private data toggle registers
 -No dedicated DMA interrupt line

Bug: 158724613
Signed-off-by: Min Guo <min.guo@mediatek.com>
Signed-off-by: Yonglong Wu <yonglong.wu@mediatek.com>
Signed-off-by: Bin Liu <b-liu@ti.com>
Link: https://lore.kernel.org/r/20200115132547.364-24-b-liu@ti.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Macpaul Lin <macpaul.lin@mediatek.com>
(cherry picked from commit 0990366bab3c6afb93b276106e1e24d4bc69db7b)
Change-Id: I5c54486be8a6b70ac99e9ae57c0a601c01612ff8
2020-06-11 19:20:26 +08:00
Min Guo
b46bf63177 UPSTREAM: usb: musb: Add musb_clearb/w() interface
Delete the const attribute of addr parameter in readb/w/l hooks, these
changes are for implementing clearing W1C registers.
Replace musb_readb/w with musb_clearb/w to clear the interrupt status.

While at here, change some unsigned type to u32 to fix checkpatch.pl
warnings.

Bug: 158724613
Signed-off-by: Min Guo <min.guo@mediatek.com>
[b-liu@ti.com: fix checkpatch.pl warnings.]
Signed-off-by: Bin Liu <b-liu@ti.com>
Link: https://lore.kernel.org/r/20200115132547.364-23-b-liu@ti.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Macpaul Lin <macpaul.lin@mediatek.com>
(cherry picked from commit 9c93d7fd464e7aad59c2afc261f80e6e0fbe2ca9)
Change-Id: I9258a28000294738a27b06229d9e12b32f96ad6c
2020-06-11 19:20:20 +08:00
Min Guo
4f79e7c304 UPSTREAM: usb: musb: Add noirq type of dma create interface
Add noirq type of dma create interface for platform which do not
have dedicated DMA interrupt line, move musbhsdma macro definition
to musb_dma.h

Bug: 158724613
Signed-off-by: Min Guo <min.guo@mediatek.com>
Signed-off-by: Bin Liu <b-liu@ti.com>
Link: https://lore.kernel.org/r/20200115132547.364-22-b-liu@ti.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Macpaul Lin <macpaul.lin@mediatek.com>
(cherry picked from commit edce61776c7e212d8b3d61e69afe7672efbacb04)
Change-Id: Ic40df362681d1c2d8ab1249246241c83655b5a3a
2020-06-11 19:20:14 +08:00
Min Guo
b62eca39e8 UPSTREAM: usb: musb: Add get/set toggle hooks
Add get/set toggle hooks in struct musb_io and struct musb_platform_ops
for special platform; remove function musb_save_toggle, use the set/get
callback to handle toggle.

Bug: 158724613
Signed-off-by: Min Guo <min.guo@mediatek.com>
Signed-off-by: Bin Liu <b-liu@ti.com>
Link: https://lore.kernel.org/r/20200115132547.364-21-b-liu@ti.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Macpaul Lin <macpaul.lin@mediatek.com>
(cherry picked from commit fe3bbd6b383fbc62128fd1fe850105080cb4c9da)
Change-Id: I1f960a519288d474e9fc00d779415c83ef190d87
2020-06-11 19:20:07 +08:00
Min Guo
d7056b4731 UPSTREAM: dt-bindings: usb: musb: Add support for MediaTek musb controller
This adds support for MediaTek musb controller in
host, peripheral and otg mode.

Bug: 158724613
Signed-off-by: Min Guo <min.guo@mediatek.com>
Reviewed-by: Rob Herring <robh@kernel.org>
Signed-off-by: Bin Liu <b-liu@ti.com>
Link: https://lore.kernel.org/r/20200115132547.364-20-b-liu@ti.com
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Signed-off-by: Macpaul Lin <macpaul.lin@mediatek.com>
(cherry picked from commit 8d817d79349d953c8b05559e4cfd448785550e78)
Change-Id: I639f5e96c88d58bb035b1f86b837c47dce48f835
2020-06-11 19:19:47 +08:00
Veerabhadrarao Badiganti
930e70e995 mmc: sdhci-msm: Ensure larger discard size
Discard size calculation uses max_busy_timeout in deriving the
value. So, we can ensure a larger discard size by setting this
timeout as zero. If this timeout is zero then it takes max erase
timeout vlaue (which is 60sec) in the discard size calculation.

And qcom SDHC can support h/w based busy detection, So
we need set MMC_CAP_NEED_RSP_BUSY capability flag.

Since Qcom SDHC can support h/w based busy detection for R1b
commands, this timeout value is not relevant for us.
Setting it zero won't affect in any operation provied
MMC_CAP_NEED_RSP_BUSY cap is set.

Change-Id: Ib20a1e536d2e24cf8de1e927715a8a3aa7f0ae27
Signed-off-by: Veerabhadrarao Badiganti <vbadigan@codeaurora.org>
2020-06-11 16:44:03 +05:30
Patrick Daly
8fbac439e0 mm: Fix sleeping while atomic during speculative page fault
A speculative page fault may race with a call to free_pagetables. If
free_pagetables is called first, *(vmf->pmd) may be empty.

__might_sleep()
__alloc_pages_nodemask()
pte_alloc_one(inline)
__pte_alloc()
pte_alloc_one_map(inline)
alloc_set_pte()
filemap_map_pages()
do_fault_around(inline)
do_read_fault(inline)
do_fault(inline)
handle_pte_fault()
mem_cgroup_exit_user_fault(inline)
__handle_speculative_fault()
do_page_fault()

As filemap_map_pages() holds an rcu_lock(), this triggers a
sleeping-while-atomic BUG(). As free_pagetables has already been called,
it is also a memory leak.

Fix this by skipping to pte_map_lock() to allow spf to detect that
the vma has changed, and a normal page fault should be taken instead.

Change-Id: I121ca4be99c908656db3a1dc88cfb3b64f01e2fb
Signed-off-by: Patrick Daly <pdaly@codeaurora.org>
Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org>
2020-06-11 16:02:09 +05:30
Laurent Dufour
c41742ec7c mm: don't do swap readahead during speculative page fault
Vinayak Menon faced a panic because one thread was page faulting a page in
swap, while another one was mprotecting a part of the VMA leading to a VMA
split.
This raise a panic in swap_vma_readahead() because the VMA's boundaries
were not more matching the faulting address.

To avoid this, if the page is not found in the swap, the speculative page
fault is aborted to retry a regular page fault.

Change-Id: Ia9d99fb5fde7bd89f38966838d115b6c8c15c9db
Signed-off-by: Laurent Dufour <ldufour@linux.vnet.ibm.com>
Patch-mainline: linux-mm @ Mon, 5 Nov 2018 18:43:01 +0100
[vinmenon@codeaurora.org: a minor 80+ line fix]
Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org>
2020-06-11 16:02:09 +05:30
Laurent Dufour
a9e3a1ab5d mm: protect against PTE changes done by dup_mmap()
Vinayak Menon and Ganesh Mahendran reported that the following scenario may
lead to thread being blocked due to data corruption:

    CPU 1                   CPU 2                    CPU 3
    Process 1,              Process 1,               Process 1,
    Thread A                Thread B                 Thread C

    while (1) {             while (1) {              while(1) {
    pthread_mutex_lock(l)   pthread_mutex_lock(l)    fork
    pthread_mutex_unlock(l) pthread_mutex_unlock(l)  }
    }                       }

In the details this happens because :

    CPU 1                CPU 2                       CPU 3
    fork()
    copy_pte_range()
      set PTE rdonly
    got to next VMA...
     .                   PTE is seen rdonly          PTE still writable
     .                   thread is writing to page
     .                   -> page fault
     .                     copy the page             Thread writes to page
     .                      .                        -> no page fault
     .                     update the PTE
     .                     flush TLB for that PTE
   flush TLB                                        PTE are now rdonly

So the write done by the CPU 3 is interfering with the page copy operation
done by CPU 2, leading to the data corruption.

To avoid this we mark all the VMA involved in the COW mechanism as changing
by calling vm_write_begin(). This ensures that the speculative page fault
handler will not try to handle a fault on these pages.
The marker is set until the TLB is flushed, ensuring that all the CPUs will
now see the PTE as not writable.
Once the TLB is flush, the marker is removed by calling vm_write_end().

The variable last is used to keep tracked of the latest VMA marked to
handle the error path where part of the VMA may have been marked.

Change-Id: I3fe07109e27d8f77c9b435053567fe5c287703aa
Reported-by: Ganesh Mahendran <opensource.ganesh@gmail.com>
Reported-by: Vinayak Menon <vinmenon@codeaurora.org>
Signed-off-by: Laurent Dufour <ldufour@linux.ibm.com>
Link: https://www.spinics.net/lists/linux-mm/msg171207.html
Patch-mainline: linux-mm@ Fri, 18 Jan 2019 17:24:16
Signed-off-by: Charan Teja Reddy <charante@codeaurora.org>
Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org>
2020-06-11 16:02:08 +05:30
Mahendran Ganesh
ad3023acf9 arm64/mm: add speculative page fault
This patch enables the speculative page fault on the arm64
architecture.

I completed spf porting in 4.9. From the test result,
we can see app launching time improved by about 10% in average.
For the apps which have more than 50 threads, 15% or even more
improvement can be got.

Signed-off-by: Ganesh Mahendran <opensource.ganesh@gmail.com>
Change-Id: Ib7c8b2e354800b5023e6c6400448a6d40aaf89c8
Patch-mainline: linux-mm @ Wed, 2 May 2018 15:54:32
[vinmenon@codeaurora.org: remove the speculative fault perf counter]
Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org>
[charante@codeaurora.org: trivial merge conflict fixes]
Signed-off-by: Charan Teja Reddy <charante@codeaurora.org>
2020-06-11 16:02:08 +05:30
Mahendran Ganesh
bcca0756b3 arm64/mm: define ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT
Set ARCH_SUPPORTS_SPECULATIVE_PAGE_FAULT for arm64. This
enables Speculative Page Fault handler.

Change-Id: Ic55bd1993b3af78ddac25867c5f422861079bb27
Signed-off-by: Ganesh Mahendran <opensource.ganesh@gmail.com>
Patch-mainline: linux-mm @ Wed, 2 May 2018 15:54:31
Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org>
2020-06-11 16:02:07 +05:30
Laurent Dufour
2842d9234e mm: add speculative page fault vmstats
Add speculative_pgfault vmstat counter to count successful speculative page
fault handling.

Also fixing a minor typo in include/linux/vm_event_item.h.

Change-Id: I0d3f3dc5195e1156d4b8edf83aff9d8d85904e8e
Signed-off-by: Laurent Dufour <ldufour@linux.vnet.ibm.com>
Patch-mainline: linux-mm @ Tue, 17 Apr 2018 16:33:29
Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org>
2020-06-11 16:02:07 +05:30
Laurent Dufour
f556cd74a7 mm: speculative page fault handler return VMA
When the speculative page fault handler is returning VM_RETRY, there is a
chance that VMA fetched without grabbing the mmap_sem can be reused by the
legacy page fault handler.  By reusing it, we avoid calling find_vma()
again. To achieve, that we must ensure that the VMA structure will not be
freed in our back. This is done by getting the reference on it (get_vma())
and by assuming that the caller will call the new service
can_reuse_spf_vma() once it has grabbed the mmap_sem.

can_reuse_spf_vma() is first checking that the VMA is still in the RB tree
, and then that the VMA's boundaries matched the passed address and release
the reference on the VMA so that it can be freed if needed.

In the case the VMA is freed, can_reuse_spf_vma() will have returned false
as the VMA is no more in the RB tree.

In the architecture page fault handler, the call to the new service
reuse_spf_or_find_vma() should be made in place of find_vma(), this will
handle the check on the spf_vma and if needed call find_vma().

Signed-off-by: Laurent Dufour <ldufour@linux.vnet.ibm.com>
Change-Id: Ia56dcf807e8bddf6788fd696dd80372db35476f0
Patch-mainline: linux-mm @ Tue, 17 Apr 2018 16:33:28
[vinmenon@codeaurora.org: trivial merge conflicts]
Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org>
2020-06-11 16:02:06 +05:30
Laurent Dufour
52e2e1f805 mm: adding speculative page fault failure trace events
This patch a set of new trace events to collect the speculative page fault
event failures.

Signed-off-by: Laurent Dufour <ldufour@linux.vnet.ibm.com>
Change-Id: I9dafba293edf40bdad4ae241d105ecdfb42579c1
Patch-mainline: linux-mm @ Tue, 17 Apr 2018 16:33:25
[vinmenon@codeaurora.org: trivial merge conflicts]
Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org>
2020-06-11 16:02:06 +05:30
Peter Zijlstra
6e9deb2ea7 mm: provide speculative fault infrastructure
Provide infrastructure to do a speculative fault (not holding
mmap_sem).

The not holding of mmap_sem means we can race against VMA
change/removal and page-table destruction. We use the SRCU VMA freeing
to keep the VMA around. We use the VMA seqcount to detect change
(including umapping / page-table deletion) and we use gup_fast() style
page-table walking to deal with page-table races.

Once we've obtained the page and are ready to update the PTE, we
validate if the state we started the fault with is still valid, if
not, we'll fail the fault with VM_FAULT_RETRY, otherwise we update the
PTE and we're done.

Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>

[Manage the newly introduced pte_spinlock() for speculative page
 fault to fail if the VMA is touched in our back]
[Rename vma_is_dead() to vma_has_changed() and declare it here]
[Fetch p4d and pud]
[Set vmd.sequence in __handle_mm_fault()]
[Abort speculative path when handle_userfault() has to be called]
[Add additional VMA's flags checks in handle_speculative_fault()]
[Clear FAULT_FLAG_ALLOW_RETRY in handle_speculative_fault()]
[Don't set vmf->pte and vmf->ptl if pte_map_lock() failed]
[Remove warning comment about waiting for !seq&1 since we don't want
 to wait]
[Remove warning about no huge page support, mention it explictly]
[Don't call do_fault() in the speculative path as __do_fault() calls
 vma->vm_ops->fault() which may want to release mmap_sem]
[Only vm_fault pointer argument for vma_has_changed()]
[Fix check against huge page, calling pmd_trans_huge()]
[Use READ_ONCE() when reading VMA's fields in the speculative path]
[Explicitly check for __HAVE_ARCH_PTE_SPECIAL as we can't support for
 processing done in vm_normal_page()]
[Check that vma->anon_vma is already set when starting the speculative
 path]
[Check for memory policy as we can't support MPOL_INTERLEAVE case due to
 the processing done in mpol_misplaced()]
[Don't support VMA growing up or down]
[Move check on vm_sequence just before calling handle_pte_fault()]
[Don't build SPF services if !CONFIG_SPECULATIVE_PAGE_FAULT]
[Add mem cgroup oom check]
[Use READ_ONCE to access p*d entries]
[Replace deprecated ACCESS_ONCE() by READ_ONCE() in vma_has_changed()]
[Don't fetch pte again in handle_pte_fault() when running the speculative
 path]
[Check PMD against concurrent collapsing operation]
[Try spin lock the pte during the speculative path to avoid deadlock with
 other CPU's invalidating the TLB and requiring this CPU to catch the
 inter processor's interrupt]
[Move define of FAULT_FLAG_SPECULATIVE here]
[Introduce __handle_speculative_fault() and add a check against
 mm->mm_users in handle_speculative_fault() defined in mm.h]
Signed-off-by: Laurent Dufour <ldufour@linux.vnet.ibm.com>
Change-Id: I45cbe79a96c7aa234cace421a36550e7fb0b9368
Patch-mainline: linux-mm @ Tue, 17 Apr 2018 16:33:24
[vinmenon@codeaurora.org: fix !CONFIG_NUMA build +
checkpatch fixes]
Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org>
[charante@codeaurora.org: trivial merge conflicts]
Signed-off-by: Charan Teja Reddy <charante@codeaurora.org>
2020-06-11 16:02:05 +05:30
Laurent Dufour
9db216b5c9 mm: protect mm_rb tree with a rwlock
This change is inspired by the Peter's proposal patch [1] which was
protecting the VMA using SRCU. Unfortunately, SRCU is not scaling well in
that particular case, and it is introducing major performance degradation
due to excessive scheduling operations.

To allow access to the mm_rb tree without grabbing the mmap_sem, this patch
is protecting it access using a rwlock.  As the mm_rb tree is a O(log n)
search it is safe to protect it using such a lock.  The VMA cache is not
protected by the new rwlock and it should not be used without holding the
mmap_sem.

To allow the picked VMA structure to be used once the rwlock is released, a
use count is added to the VMA structure. When the VMA is allocated it is
set to 1.  Each time the VMA is picked with the rwlock held its use count
is incremented. Each time the VMA is released it is decremented. When the
use count hits zero, this means that the VMA is no more used and should be
freed.

This patch is preparing for 2 kind of VMA access :
 - as usual, under the control of the mmap_sem,
 - without holding the mmap_sem for the speculative page fault handler.

Access done under the control the mmap_sem doesn't require to grab the
rwlock to protect read access to the mm_rb tree, but access in write must
be done under the protection of the rwlock too. This affects inserting and
removing of elements in the RB tree.

The patch is introducing 2 new functions:
 - vma_get() to find a VMA based on an address by holding the new rwlock.
 - vma_put() to release the VMA when its no more used.
These services are designed to be used when access are made to the RB tree
without holding the mmap_sem.

When a VMA is removed from the RB tree, its vma->vm_rb field is cleared and
we rely on the WMB done when releasing the rwlock to serialize the write
with the RMB done in a later patch to check for the VMA's validity.

When free_vma is called, the file associated with the VMA is closed
immediately, but the policy and the file structure remained in used until
the VMA's use count reach 0, which may happens later when exiting an
in progress speculative page fault.

[1] https://patchwork.kernel.org/patch/5108281/

Change-Id: I9ecc922b8efa4b28975cc6a8e9531284c24ac14e
Cc: Peter Zijlstra (Intel) <peterz@infradead.org>
Cc: Matthew Wilcox <willy@infradead.org>
Signed-off-by: Laurent Dufour <ldufour@linux.vnet.ibm.com>
Patch-mainline: linux-mm @ Tue, 17 Apr 2018 16:33:23
[vinmenon@codeaurora.org: fix the return of put_vma]
Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org>
Signed-off-by: Charan Teja Reddy <charante@codeaurora.org>
2020-06-11 16:02:05 +05:30
Laurent Dufour
d56126780c mm: introduce __page_add_new_anon_rmap()
When dealing with speculative page fault handler, we may race with VMA
being split or merged. In this case the vma->vm_start and vm->vm_end
fields may not match the address the page fault is occurring.

This can only happens when the VMA is split but in that case, the
anon_vma pointer of the new VMA will be the same as the original one,
because in __split_vma the new->anon_vma is set to src->anon_vma when
*new = *vma.

So even if the VMA boundaries are not correct, the anon_vma pointer is
still valid.

If the VMA has been merged, then the VMA in which it has been merged
must have the same anon_vma pointer otherwise the merge can't be done.

So in all the case we know that the anon_vma is valid, since we have
checked before starting the speculative page fault that the anon_vma
pointer is valid for this VMA and since there is an anon_vma this
means that at one time a page has been backed and that before the VMA
is cleaned, the page table lock would have to be grab to clean the
PTE, and the anon_vma field is checked once the PTE is locked.

This patch introduce a new __page_add_new_anon_rmap() service which
doesn't check for the VMA boundaries, and create a new inline one
which do the check.

When called from a page fault handler, if this is not a speculative one,
there is a guarantee that vm_start and vm_end match the faulting address,
so this check is useless. In the context of the speculative page fault
handler, this check may be wrong but anon_vma is still valid as explained
above.

Change-Id: I72c47830181579f8c9618df879077d321653b5f1
Signed-off-by: Laurent Dufour <ldufour@linux.vnet.ibm.com>
Patch-mainline: linux-mm @ Tue, 17 Apr 2018 16:33:22
[vinmenon@codeaurora.org: trivial merge conflict fixes +
checkpatch fixes]
Signed-off-by: Vinayak Menon <vinmenon@codeaurora.org>
2020-06-11 16:02:04 +05:30