RE: How to best solve fragmentation issues

["Jens Andersson" <lists@...>]

Thanks for all the good suggestions about this problem. Writing a simple
allocator like suggested seems like a decent solution. I did try another
approach first though: I remade my program so that there is a specific point
during execution where I can free all dynamic allocations. After
implementing that I still had fragmentation issues, so after logging all
remaining allocations between two runs a bunch of LUA allocations showed up.
Since I doubt there are memory leaks in there, I assume that I do something
wrong, but a call to lua_close() should free up all dynamic allocations,
shouldn't it? I also tried a lua_gc(L, LUA_GCCOLLECT, 0) call before closing
the state, but that didn't affect anything. 

Thanks,

/Jens

-----Original Message-----
From: lua-bounces@bazar2.conectiva.com.br
[mailto:lua-bounces@bazar2.conectiva.com.br] On Behalf Of Stephen Kellett
Sent: den 22 februari 2008 17:13
To: Lua list
Subject: Re: How to best solve fragmentation issues

Jens Andersson wrote:
> separate heap. I'm not really looking forward to implement a separate
> memory-manager, but this way the LUA allocations won't interfere with the
> rest of the memory.
> 
> Is there a better way to solve this?

You may be better off writing a C extension that provides an API that 
simply allocates one large chunk of memory and then 
allocates/deallocates that memory explicitly. Four functions should do it:
	Create heap
	allocate from heap
	deallocate from heap
	Destroy heap

This removes you from the problem of having to write a full blown memory 
manager to replace the Lua one, plus the problems of making your custom 
memory manager as fast as the general purpose one (your simple cases may 
be faster, but the nasty cases will most likley be worse, or just fail).

It also means that if your images are all the same size or you know the 
largest size of the images and how many images you'll need at any one 
time you can ensure your heap has this space and also ensure that your 
allocate()deallocate() functions prevent fragmentation.

I know I'm advocating explicit memory management in a GC'd world. But 
for things like this its almost certainly a better bet. I've come across 
some similar cases in the Windows world (where you can force the result 
with VirtualAlloc/VirtualFree or 
HeapCreate/HeapDestroy/HeapAlloc/HeapFree). The C allocator will never 
give you the control so that you can guarantee the result.

Stephen