lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]


I applied your new patch. Running the garbage collector every time a
new allocation is needed clearly decreases the required memory amount,
but also slows the script a lot. An example on 'life.lua' with TLSF
(now it runs with TLSF; I think it was a problem with my statistics
module, not with your patch):

TLSF without GC being called by the allocation function: needs 182,000
bytes (this is the reference).
TLSF with GC running before each new allocation: needs 67,000 bytes of
RAM (an excellent value, but very slow)
TLSF with GC running only after a new allocation couldn't be
satisfied: needs 180,000 byes of RAM (not much of an improvement)
TLSF with GC being called when a limit of 64000 bytes is reached:
needs 72000 bytes (again an excellent value, and the performance seems
very good).

This is only one test, so it's hard to extrapolate, but the results
seem interesting, and I can already presume that running the GC all
the time will not have much practical value, since it slows Lua a lot.
Also, the 64000 limit on the last case presented above is quite
interesting, because once you move to other values (upper or lower)
you'll get a 'not enough memory' error. I was only able to run the
script in 72000 bytes with the 'magical' 64000 limit, and not anything
else. I can't explain this yet. Also, when I try to run
'factorial.lua' with the 'always call GC' method (TLSF limited to 39k
of RAM) I get a segfault with this stack trace:

#0  0x00000000 in ?? ()
#1  0x080504a6 in luaD_precall (L=0x80755a4, func=0x807581c,
nresults=-1) at lua-5.1.3/src/ldo.c:319
#2  0x0805e86c in luaV_execute (L=0x80755a4, nexeccalls=2) at
lua-5.1.3/src/lvm.c:610
#3  0x08050706 in luaD_call (L=0x80755a4, func=0x80757f8, nResults=-1)
at lua-5.1.3/src/ldo.c:377
#4  0x0804b33f in f_call (L=0x80755a4, ud=0xfff54154) at
lua-5.1.3/src/lapi.c:801
#5  0x0804f8a9 in luaD_rawrunprotected (L=0x80755a4, f=0x804b315
<f_call>, ud=0xfff54154) at lua-5.1.3/src/ldo.c:116
#6  0x08050a62 in luaD_pcall (L=0x80755a4, func=0x804b315 <f_call>,
u=0xfff54154, old_top=48, ef=36) at lua-5.1.3/src/ldo.c:463
#7  0x0804b3d3 in lua_pcall (L=0x80755a4, nargs=0, nresults=-1,
errfunc=2) at lua-5.1.3/src/lapi.c:822
#8  0x0806b4f9 in docall (L=0x80755a4, narg=0, clear=0) at
lua-5.1.3/src/lua.c:102
#9  0x0806bd56 in handle_script (L=0x80755a4, argv=0xfff54514, n=1) at
lua-5.1.3/src/lua.c:250
#10 0x0806c225 in pmain (L=0x80755a4) at lua-5.1.3/src/lua.c:362
#11 0x080504a6 in luaD_precall (L=0x80755a4, func=0x80757d4,
nresults=0) at lua-5.1.3/src/ldo.c:319
#12 0x080506ef in luaD_call (L=0x80755a4, func=0x80757d4, nResults=0)
at lua-5.1.3/src/ldo.c:376
#13 0x0804b4b6 in f_Ccall (L=0x80755a4, ud=0xfff543cc) at
lua-5.1.3/src/lapi.c:847
#14 0x0804f8a9 in luaD_rawrunprotected (L=0x80755a4, f=0x804b403
<f_Ccall>, ud=0xfff543cc) at lua-5.1.3/src/ldo.c:116
#15 0x08050a62 in luaD_pcall (L=0x80755a4, func=0x804b403 <f_Ccall>,
u=0xfff543cc, old_top=12, ef=0) at lua-5.1.3/src/ldo.c:463
#16 0x0804b504 in lua_cpcall (L=0x80755a4, func=0x806c0a1 <pmain>,
ud=0xfff54400) at lua-5.1.3/src/lapi.c:857
#17 0x0806c315 in lua_main (argc=2, argv=0xfff54514) at lua-5.1.3/src/lua.c:396
#18 0x0806d1b3 in main (argc=6, argv=0xfff54504) at luastat.c:493

(I applied your latest patches). I still don't know if the error is in
my code or in your code; however, I only modified 'lua.c' very briefly
for my statistics module, and I find it hard to believe that my
modification could lead to a stack trace like the one above :)
Also, in your new 'memlimit.c' file, I noticed that you don't use the
'collecting' guard variable anymore. Is this an omission, or it's a
consequence of the patch?

On Sun, May 4, 2008 at 6:18 AM, Robert G. Jakabosky
<bobby@sharedrealm.com> wrote:
> Updated patch attached "emergency_gc_fixes.patch".
>
>  The patch fixes some bugs where the garbage collector (GC) will free an object
>  that is being created and hasn't been put onto the Lua stack or added to a
>  table yet.  Also it protects the GC from recursive calls
>  (allocator->GC->allocator->GC or GC->allocator->GC).
>
>  Since this patch only fixes bug related to running the garbage collector from
>  the allocator function, I only recommend using it if you need to restrict the
>  amount of memory used by a Lua script.  Also there maybe more bugs that I
>  haven't found yet, so use at your own risk.
>
>  Also attached is a new version of lua_memlimit.c + some lua scripts for stress
>  testing.  The new version adds a lot of checks and debug messages to the
>  allocator for debugging GC bugs.  There are some #define's at the top to
>  control how many debug messages are enabled and what checks to run.  Does
>  anyone know a good set of Lua scripts for testing all Lua features?  Do the
>  scripts that come with lua 5.1.3 test all the language features?
>
>  Summery of bug that the patch fixes (warning long read):
>  Function "lua_setfield" in lapi.c
>   This function created a Lua string and stored it in a variable on the C
>  stack, had to move the string to the Lua stack so the GC wouldn't free the
>  string before the key/value was added to the table.  This is something that
>  can effect other C libraries that create temporary Lua objects on the C
>  stack.  If another allocation happens before the object is put on the Lua
>  stack or added to a table the GC might free the object.
>
>  Function "lua_gc" in lapi.c
>   Added protection from recursive LUA_GCCOLLECT calls.  Also LUA_GCSTOP &
>  LUA_GCRESTART can be used to stop a full GC.
>
>  Function "f_parser" in ldo.c
>   I haven't look to deeply into the Lua parser yet, but it looks like there
>  are a lot of places where objects are created and not placed somewhere that
>  the GC can find.  So for now I just stop the GC before the parser runs, then
>  restart it after.
>
>  Function "checkSizes" in lgc.c
>   This function was shrinking a temp. buffer used to concatenate strings.  Now
>  it will not shrink the size below the in use size needed by the string
>  concat. function.
>
>  Function "luaE_newthread" in lstate.c
>   The new lua_State object wasn't fully allocated before a link to it was
>  added to the GC.  I only had to move the call to "luaC_link" to the end of
>  the function.
>
>  Function "luaH_new" in ltable.c
>   The table object wasn't fully allocated before a link to it was added to the
>  GC.  I only had to move the call to "luaC_link" to the end of the function.
>
>  Function "newlstr" in lstring.c
>   This function creates a new Lua string object and adds it to the global
>  string hashtable, it also calls "luaS_resize" to expand the string hashtable
>  when it gets too crowded.  The change needed here was to delay adding the
>  string to the hashtable until after the hashtable was resized.
>
>  Function "luaV_concat" in lvm.c
>   Same as last patch, the temp. buffer has it's "in use" size updated so the
>  GC will not shrink the buffer below the "in use" size.  The "in use" size is
>  reset to 0 after the buffer's contents are copied to a new Lua string.
>
>  --
>  Robert G. Jakabosky
>