slab,slub: don't enable interrupts during early boot

As explained by Benjamin Herrenschmidt:

  Oh and btw, your patch alone doesn't fix powerpc, because it's missing
  a whole bunch of GFP_KERNEL's in the arch code... You would have to
  grep the entire kernel for things that check slab_is_available() and
  even then you'll be missing some.

  For example, slab_is_available() didn't always exist, and so in the
  early days on powerpc, we used a mem_init_done global that is set form
  mem_init() (not perfect but works in practice). And we still have code
  using that to do the test.

Therefore, mask out __GFP_WAIT, __GFP_IO, and __GFP_FS in the slab allocators
in early boot code to avoid enabling interrupts.

Signed-off-by: Pekka Enberg <[email protected]>
This commit is contained in:
Pekka Enberg 2009-06-12 14:03:06 +03:00
commit 7e85ee0c1d
7 changed files with 47 additions and 0 deletions

View file

@ -303,6 +303,12 @@ struct kmem_list3 {
int free_touched; /* updated without locking */
};
/*
* The slab allocator is initialized with interrupts disabled. Therefore, make
* sure early boot allocations don't accidentally enable interrupts.
*/
static gfp_t slab_gfp_mask __read_mostly = SLAB_GFP_BOOT_MASK;
/*
* Need this for bootstrapping a per node allocator.
*/
@ -1654,6 +1660,14 @@ void __init kmem_cache_init(void)
*/
}
void __init kmem_cache_init_late(void)
{
/*
* Interrupts are enabled now so all GFP allocations are safe.
*/
slab_gfp_mask = __GFP_BITS_MASK;
}
static int __init cpucache_init(void)
{
int cpu;
@ -3354,6 +3368,8 @@ __cache_alloc_node(struct kmem_cache *cachep, gfp_t flags, int nodeid,
unsigned long save_flags;
void *ptr;
flags &= slab_gfp_mask;
lockdep_trace_alloc(flags);
if (slab_should_failslab(cachep, flags))
@ -3434,6 +3450,8 @@ __cache_alloc(struct kmem_cache *cachep, gfp_t flags, void *caller)
unsigned long save_flags;
void *objp;
flags &= slab_gfp_mask;
lockdep_trace_alloc(flags);
if (slab_should_failslab(cachep, flags))