mm: memcontrol: per-lruvec stats infrastructure

lruvecs are at the intersection of the NUMA node and memcg, which is the
scope for most paging activity.

Introduce a convenient accounting infrastructure that maintains
statistics per node, per memcg, and the lruvec itself.

Then convert over accounting sites for statistics that are already
tracked in both nodes and memcgs and can be easily switched.

[[email protected]: fix crash in the new cgroup stat keeping code]
  Link: http://lkml.kernel.org/r/[email protected]
[[email protected]: don't track uncharged pages at all
  Link: http://lkml.kernel.org/r/[email protected]
[[email protected]: add missing free_percpu()]
  Link: http://lkml.kernel.org/r/[email protected]
[[email protected]: hexagon: fix build error caused by include file order]
  Link: http://lkml.kernel.org/r/[email protected]
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Johannes Weiner <[email protected]>
Signed-off-by: Guenter Roeck <[email protected]>
Acked-by: Vladimir Davydov <[email protected]>
Cc: Josef Bacik <[email protected]>
Cc: Michal Hocko <[email protected]>
Cc: Rik van Riel <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
This commit is contained in:
Johannes Weiner 2017-07-06 15:40:52 -07:00 • committed by Linus Torvalds
commit 00f3ca2c2d
9 changed files with 238 additions and 55 deletions

View file

@ -1145,8 +1145,7 @@ void page_add_file_rmap(struct page *page, bool compound)
if (!atomic_inc_and_test(&page->_mapcount))
goto out;
}
__mod_node_page_state(page_pgdat(page), NR_FILE_MAPPED, nr);
mod_memcg_page_state(page, NR_FILE_MAPPED, nr);
__mod_lruvec_page_state(page, NR_FILE_MAPPED, nr);
out:
unlock_page_memcg(page);
}
@ -1181,12 +1180,11 @@ static void page_remove_file_rmap(struct page *page, bool compound)
}
/*
* We use the irq-unsafe __{inc|mod}_zone_page_state because
* We use the irq-unsafe __{inc|mod}_lruvec_page_state because
* these counters are not modified in interrupt context, and
* pte lock(a spinlock) is held, which implies preemption disabled.
*/
__mod_node_page_state(page_pgdat(page), NR_FILE_MAPPED, -nr);
mod_memcg_page_state(page, NR_FILE_MAPPED, -nr);
__mod_lruvec_page_state(page, NR_FILE_MAPPED, -nr);
if (unlikely(PageMlocked(page)))
clear_page_mlock(page);