Dave Hansen
534acc057b
lib: flexible array implementation ...
Once a structure goes over PAGE_SIZE*2, we see occasional allocation
failures. Some people have chosen to switch over to things like vmalloc()
that will let them keep array-like access to such a large structures.
But, vmalloc() has plenty of downsides.
Here's an alternative. I think it's what Andrew was suggesting here:
http://lkml.org/lkml/2009/7/2/518
I call it a flexible array. It does all of its work in PAGE_SIZE bits, so
never does an order>0 allocation. The base level has
PAGE_SIZE-2*sizeof(int) bytes of storage for pointers to the second level.
So, with a 32-bit arch, you get about 4MB (4183112 bytes) of total
storage when the objects pack nicely into a page. It is half that on
64-bit because the pointers are twice the size. There's a table detailing
this in the code.
There are kerneldocs for the functions, but here's an
overview:
flex_array_alloc() - dynamically allocate a base structure
flex_array_free() - free the array and all of the
second-level pages
flex_array_free_parts() - free the second-level pages, but
not the base (for static bases)
flex_array_put() - copy into the array at the given index
flex_array_get() - copy out of the array at the given index
flex_array_prealloc() - preallocate the second-level pages
between the given indexes to
guarantee no allocs will occur at
put() time.
We could also potentially just pass the "element_size" into each of the
API functions instead of storing it internally. That would get us one
more base pointer on 32-bit.
I've been testing this by running it in userspace. The header and patch
that I've been using are here, as well as the little script I'm using to
generate the size table which goes in the kerneldocs.
http://sr71.net/~dave/linux/flexarray/
[akpm@linux-foundation.org: coding-style fixes]
Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2009-07-29 19:10:36 -07:00
lzo: use get/put_unaligned_* helpers
2008-07-25 10:53:26 -07:00
bzip2/lzma: library support for gzip, bzip2 and lzma decompression
2009-01-04 15:53:34 -08:00
lib: export generic atomic64_t functions
2009-07-29 19:10:35 -07:00
rtc: BCD codeshrink
2008-07-24 10:47:33 -07:00
bitmap: fix end condition in bitmap_find_free_region
2009-03-12 19:32:51 -07:00
lib: export bitrev16
2008-06-06 11:29:10 -07:00
allow bug table entries to use relative pointers (and use it on x86-64)
2008-12-16 18:40:32 +01:00
oops handling: ensure that any oops is flushed to the mtdoops console
2009-01-06 15:59:11 -08:00
lib/checksum.c: fix endianess bug
2009-06-19 14:58:13 +02:00
generic, memparse(): constify argument
2008-07-28 15:05:23 +02:00
x86: remove some alloc_bootmem_cpumask_var calling
2009-06-11 19:27:07 +03:00
[SCSI] lib: Add support for the T10 (SCSI) Data Integrity Field CRC
2008-07-12 08:22:32 -05:00
lockdep: warn about lockdep disabling after kernel taint
2009-04-12 16:10:51 +02:00
debugobjects: delay free of internal objects
2009-03-17 12:28:30 +01:00
atomic: only take lock when the counter drops to zero on UP as well
2009-06-16 19:47:47 -07:00
bzip2/lzma: don't stop search at first unconfigured compression
2009-02-19 13:36:55 -08:00
kmemtrace, kbuild: fix slab.h dependency problem in lib/decompress_bunzip2.c
2009-04-03 12:22:45 +02:00
kmemtrace, kbuild: fix slab.h dependency problem in lib/decompress_inflate.c
2009-04-03 12:21:36 +02:00
kmemtrace, kbuild: fix slab.h dependency problem in lib/decompress_unlzma.c
2009-04-03 12:22:59 +02:00
[POWERPC] devres: Add devm_ioremap_prot()
2008-05-05 16:47:14 +10:00
add an inlined version of iter_div_u64_rem
2008-06-12 10:47:58 +02:00
dma-debug: Fix the overlap() function to be correct and readable
2009-07-10 22:18:45 +02:00
Dynamic debug: fix typo: -/->
2009-07-28 13:45:22 -07:00
module: trim exception table on init free.
2009-06-12 21:47:04 +09:30
Remove remaining unwinder code
2009-01-06 15:59:11 -08:00
bitmap: find_last_bit()
2009-01-01 10:12:19 +10:30
bitops: remove "optimizations"
2008-04-29 08:11:16 -07:00
lib: flexible array implementation
2009-07-29 19:10:36 -07:00
lib: add lib/gcd.c
2009-06-18 13:04:05 -07:00
lib/genalloc.c: remove unmatched write_lock() in gen_pool_destroy
2009-06-16 19:47:53 -07:00
hexdump: remove the trailing space
2009-06-16 19:47:51 -07:00
cgroup: CSS ID support
2009-04-02 19:04:53 -07:00
inflate: refactor inflate malloc code
2008-07-25 10:53:28 -07:00
Use WARN() in lib/
2008-07-26 12:00:07 -07:00
introduce generic iommu_num_pages function
2008-10-16 11:21:33 -07:00
CRED: Rename is_single_threaded() to is_wq_single_threaded()
2008-11-14 10:39:21 +11:00
lib: Provide generic atomic64_t implementation
2009-06-15 13:27:38 +10:00
kmemleak: Allow the early log buffer to be configurable.
2009-06-25 10:16:13 +01:00
kgdb: remove the requirement for CONFIG_FRAME_POINTER
2008-08-01 08:39:34 -05:00
kmemcheck: make kconfig accessible for other architectures
2009-06-15 15:49:17 +02:00
sched: TIF_NEED_RESCHED -> need_reshed() cleanup
2009-03-06 12:48:55 +01:00
driver core: Remove completion from struct klist_node
2009-01-06 10:44:30 -08:00
kobject: make kset_create check kobject_set_name return value
2009-06-15 21:30:24 -07:00
driver core: allow non-root users to listen to uevents
2009-04-16 16:17:09 -07:00
libcrc32c: Fix "crc32c undefined" compilation error
2008-12-25 11:01:42 +11:00
list debugging: use WARN() instead of BUG()
2008-07-25 10:53:29 -07:00
lmb: Rework lmb_dump_all() output
2009-02-11 13:38:00 +11:00
locking: rename trace_softirq_[enter|exit] => lockdep_softirq_[enter|exit]
2009-03-13 01:32:36 +01:00
lib: flexible array implementation
2009-07-29 19:10:36 -07:00
Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/kaber/nf-next-2.6
2009-03-26 22:45:23 -07:00
vfs: Use const for kernel parser table
2008-10-13 10:10:37 -07:00
Merge branch 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2009-01-06 17:10:04 -08:00
Use WARN() in lib/
2008-07-26 12:00:07 -07:00
lib: fix sparse shadowed variable warning
2009-01-06 15:59:11 -08:00
Merge branch 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip
2009-01-06 17:10:04 -08:00
lib: do code optimization for radix_tree_lookup() and radix_tree_lookup_slot()
2009-06-16 19:47:49 -07:00
random32: seeding improvement
2008-07-30 16:29:19 -07:00
__ratelimit() cpu flags can't be static
2008-07-28 16:30:20 -07:00
lib: isolate rational fractions helper function
2009-06-11 08:51:08 -07:00
rb_tree: remove redundant if()-condition in rb_erase()
2009-06-16 19:47:56 -07:00
scatterlist: make sure sg_miter_next() doesn't return 0 sized mappings
2009-04-22 08:35:09 +02:00
lib: generic show_mem()
2008-07-26 12:00:10 -07:00
cpumask: convert lib/smp_processor_id to new cpumask ops
2009-01-30 15:47:34 +01:00
generic swap(): lib/sort.c: rename swap to swap_func
2009-01-08 08:31:14 -08:00
Add a new sysfs_streq() string comparison function
2008-05-01 08:03:59 -07:00
[SCSI] lib: string_get_size(): don't hang on zero; no decimals on exact
2008-10-23 11:42:20 -05:00
swiotlb: change swiotlb_bus_to[phys,virt] prototypes
2009-04-08 16:18:38 +02:00
task_current_syscall
2008-07-26 12:00:10 -07:00
remove CONFIG_KMOD from lib
2008-07-22 19:24:31 +10:00
textsearch: ts_bm: support case insensitive searching in Boyer-Moore algorithm
2008-07-08 02:37:54 -07:00
textsearch: ts_fsm: return error on request for case insensitive search
2008-07-08 02:38:27 -07:00
textsearch: ts_kmp: support case insensitive searching in Knuth-Morris-Pratt algorithm
2008-07-08 02:38:09 -07:00
vsprintf: introduce %pf format specifier
2009-04-29 20:55:55 +02:00
534acc057b