memcg, slab: fix barrier usage when accessing memcg_caches

Each root kmem_cache has pointers to per-memcg caches stored in its
memcg_params::memcg_caches array.  Whenever we want to allocate a slab
for a memcg, we access this array to get per-memcg cache to allocate
from (see memcg_kmem_get_cache()).  The access must be lock-free for
performance reasons, so we should use barriers to assert the kmem_cache
is up-to-date.

First, we should place a write barrier immediately before setting the
pointer to it in the memcg_caches array in order to make sure nobody
will see a partially initialized object.  Second, we should issue a read
barrier before dereferencing the pointer to conform to the write
barrier.

However, currently the barrier usage looks rather strange.  We have a
write barrier *after* setting the pointer and a read barrier *before*
reading the pointer, which is incorrect.  This patch fixes this.

Signed-off-by: Vladimir Davydov <[email protected]>
Cc: Michal Hocko <[email protected]>
Cc: Glauber Costa <[email protected]>
Cc: Johannes Weiner <[email protected]>
Cc: Balbir Singh <[email protected]>
Cc: KAMEZAWA Hiroyuki <[email protected]>
Cc: Pekka Enberg <[email protected]>
Cc: Christoph Lameter <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
This commit is contained in:
Vladimir Davydov 2014-01-23 15:52:59 -08:00 • committed by Linus Torvalds
commit 959c8963fc
2 changed files with 21 additions and 15 deletions

View file

@ -163,9 +163,19 @@ static inline const char *cache_name(struct kmem_cache *s)
static inline struct kmem_cache *
cache_from_memcg_idx(struct kmem_cache *s, int idx)
{
struct kmem_cache *cachep;
if (!s->memcg_params)
return NULL;
return s->memcg_params->memcg_caches[idx];
cachep = s->memcg_params->memcg_caches[idx];
/*
* Make sure we will access the up-to-date value. The code updating
* memcg_caches issues a write barrier to match this (see
* memcg_register_cache()).
*/
smp_read_barrier_depends();
return cachep;
}
static inline struct kmem_cache *memcg_root_cache(struct kmem_cache *s)