Re: How to best solve fragmentation issues

[Stephen Kellett <lua@...>]

Jens Andersson wrote:
> separate heap. I'm not really looking forward to implement a separate
> memory-manager, but this way the LUA allocations won't interfere with the
> rest of the memory.
>
> Is there a better way to solve this?

You may be better off writing a C extension that provides an API that 
simply allocates one large chunk of memory and then 
allocates/deallocates that memory explicitly. Four functions should do it:
	Create heap
	allocate from heap
	deallocate from heap
	Destroy heap

This removes you from the problem of having to write a full blown memory 
manager to replace the Lua one, plus the problems of making your custom 
memory manager as fast as the general purpose one (your simple cases may 
be faster, but the nasty cases will most likley be worse, or just fail).

It also means that if your images are all the same size or you know the 
largest size of the images and how many images you'll need at any one 
time you can ensure your heap has this space and also ensure that your 
allocate()deallocate() functions prevent fragmentation.

I know I'm advocating explicit memory management in a GC'd world. But 
for things like this its almost certainly a better bet. I've come across 
some similar cases in the Windows world (where you can force the result 
with VirtualAlloc/VirtualFree or 
HeapCreate/HeapDestroy/HeapAlloc/HeapFree). The C allocator will never 
give you the control so that you can guarantee the result.

Stephen