lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]


Do you need to worry about the memory allocator being invoked during a
GC call, and recursively invoking lua_gc() again?

Is is possible that a _gc metamethod that might be invoked could
allocate memory in the process of running the method and freeing
up storage and causing a recursive invocation of lua_gc()?

louie

On May 3, 2008, at 1:25 PM, Bogdan Marinescu wrote:

Well, my preliminary tests show that your patch certainly makes a
difference on my test environment (by 'patch' I mean the way you call
the garbage collector when there's no more memory, not the GC patch).
As mentioned on some other thread, I'm using a small statistics module
to measure the performance and usability of some allocators in
conjunction with Lua. Using my ultra-simple chained allocator,
'speed-test.lua' needed at least 66k to run before using the patch,
now it runs happily with 39k, which is quite an improvement. I'm
having some problems making it run with TLSF; for some reason, Lua
segfaults with this allocator when your patch is applied and I don't
understand why (it runs fine with TLSF without the patch). Which is
not nice, since TLSF gave some very good results when used with Lua,
and I'm very curious about the TLFS + patch combination. In any case,
I think your patch would be useful in any resource constrained
embedded system (and most of them are resource constrained). People on
desktops will probably be reluctant to use this, since they generally
don't have problems related to memory consumption and the patch would
only slow down Lua, but I consider this to be an important step
towards making Lua embedded-friendly.
On thing that I've noticed is that setting the limit to 64k (which is
the maximum internal memory of many ARM cores out there) doesn't
really help, as the allocator is likely to return NULL because
reaching this limit. However, setting the limit to 3/4 of the actual
available memory changes the situation dramatically.

On Fri, May 2, 2008 at 11:42 PM, Robert G. Jakabosky
<bobby@sharedrealm.com> wrote:
On Friday 02, Alex Davies wrote:
Sounds interesting, I'm surprised that's the only place you ran in to any problems. Just so you know, the next version of lua features an emergency garbage collector when a request cannot be satisfied. I don't know whether this is the same as a full collectgarbage, or if it just finishes the
current phase or what.
I saw the emergency garbage collector in the feature list for Lua 5.2, after fixing this bug. Also I just read an old-post(June 2006) from the mailling list saying that there would be a problem with allocating tables if the GC
was called after a failed allocation.  I will be doing more testing.


Just out of curiousity ;) what happens if the second request fails as well and an error is thrown? I assume in this obscure case the "used" value would be incorrect - doubt it would be a problem but just something to
consider before making any assumptions about used.
If the GC call doesn't free enough memory then the allocator resets the "used" value and returns NULL (and that will cause an error to be thrown). Also I have an assert(used == 0) that is checked right after calling lua_close(L) to
make sure all allocated memory is correctly accounted for.

Here is the simplified code for the allocator:

old_used = memused; /* save old memused. */
memused -= osize;
if(nsize == 0) {
 free(ptr);
 return NULL;
}
memused += nsize;
if(nsize > osize && memused >= max_memused) {
 memused = old_size; /* restore old memused. */
 lua_gc(L, LUA_GCCOLLECT, 0);
 /* check memory usage again. */
 old_size = memused;
 memused -= osize;
 memused += nsize;
 if(memused >= max_memused) {
   /* GC wasn't able to free enough memory.  Fail allocation.*/
   memused = old_size;
   return NULL;
 }
}
return realloc(ptr, nsize);

--
Robert G. Jakabosky