[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
- Subject: Re: PATCH: fixes bug with calling garbage collector from custom lua_Alloc
- From: "Bogdan Marinescu" <bogdan.marinescu@...>
- Date: Sun, 4 May 2008 15:16:59 +0300
I applied your new patch. Running the garbage collector every time a
new allocation is needed clearly decreases the required memory amount,
but also slows the script a lot. An example on 'life.lua' with TLSF
(now it runs with TLSF; I think it was a problem with my statistics
module, not with your patch):
TLSF without GC being called by the allocation function: needs 182,000
bytes (this is the reference).
TLSF with GC running before each new allocation: needs 67,000 bytes of
RAM (an excellent value, but very slow)
TLSF with GC running only after a new allocation couldn't be
satisfied: needs 180,000 byes of RAM (not much of an improvement)
TLSF with GC being called when a limit of 64000 bytes is reached:
needs 72000 bytes (again an excellent value, and the performance seems
very good).
This is only one test, so it's hard to extrapolate, but the results
seem interesting, and I can already presume that running the GC all
the time will not have much practical value, since it slows Lua a lot.
Also, the 64000 limit on the last case presented above is quite
interesting, because once you move to other values (upper or lower)
you'll get a 'not enough memory' error. I was only able to run the
script in 72000 bytes with the 'magical' 64000 limit, and not anything
else. I can't explain this yet. Also, when I try to run
'factorial.lua' with the 'always call GC' method (TLSF limited to 39k
of RAM) I get a segfault with this stack trace:
#0 0x00000000 in ?? ()
#1 0x080504a6 in luaD_precall (L=0x80755a4, func=0x807581c,
nresults=-1) at lua-5.1.3/src/ldo.c:319
#2 0x0805e86c in luaV_execute (L=0x80755a4, nexeccalls=2) at
lua-5.1.3/src/lvm.c:610
#3 0x08050706 in luaD_call (L=0x80755a4, func=0x80757f8, nResults=-1)
at lua-5.1.3/src/ldo.c:377
#4 0x0804b33f in f_call (L=0x80755a4, ud=0xfff54154) at
lua-5.1.3/src/lapi.c:801
#5 0x0804f8a9 in luaD_rawrunprotected (L=0x80755a4, f=0x804b315
<f_call>, ud=0xfff54154) at lua-5.1.3/src/ldo.c:116
#6 0x08050a62 in luaD_pcall (L=0x80755a4, func=0x804b315 <f_call>,
u=0xfff54154, old_top=48, ef=36) at lua-5.1.3/src/ldo.c:463
#7 0x0804b3d3 in lua_pcall (L=0x80755a4, nargs=0, nresults=-1,
errfunc=2) at lua-5.1.3/src/lapi.c:822
#8 0x0806b4f9 in docall (L=0x80755a4, narg=0, clear=0) at
lua-5.1.3/src/lua.c:102
#9 0x0806bd56 in handle_script (L=0x80755a4, argv=0xfff54514, n=1) at
lua-5.1.3/src/lua.c:250
#10 0x0806c225 in pmain (L=0x80755a4) at lua-5.1.3/src/lua.c:362
#11 0x080504a6 in luaD_precall (L=0x80755a4, func=0x80757d4,
nresults=0) at lua-5.1.3/src/ldo.c:319
#12 0x080506ef in luaD_call (L=0x80755a4, func=0x80757d4, nResults=0)
at lua-5.1.3/src/ldo.c:376
#13 0x0804b4b6 in f_Ccall (L=0x80755a4, ud=0xfff543cc) at
lua-5.1.3/src/lapi.c:847
#14 0x0804f8a9 in luaD_rawrunprotected (L=0x80755a4, f=0x804b403
<f_Ccall>, ud=0xfff543cc) at lua-5.1.3/src/ldo.c:116
#15 0x08050a62 in luaD_pcall (L=0x80755a4, func=0x804b403 <f_Ccall>,
u=0xfff543cc, old_top=12, ef=0) at lua-5.1.3/src/ldo.c:463
#16 0x0804b504 in lua_cpcall (L=0x80755a4, func=0x806c0a1 <pmain>,
ud=0xfff54400) at lua-5.1.3/src/lapi.c:857
#17 0x0806c315 in lua_main (argc=2, argv=0xfff54514) at lua-5.1.3/src/lua.c:396
#18 0x0806d1b3 in main (argc=6, argv=0xfff54504) at luastat.c:493
(I applied your latest patches). I still don't know if the error is in
my code or in your code; however, I only modified 'lua.c' very briefly
for my statistics module, and I find it hard to believe that my
modification could lead to a stack trace like the one above :)
Also, in your new 'memlimit.c' file, I noticed that you don't use the
'collecting' guard variable anymore. Is this an omission, or it's a
consequence of the patch?
On Sun, May 4, 2008 at 6:18 AM, Robert G. Jakabosky
<bobby@sharedrealm.com> wrote:
> Updated patch attached "emergency_gc_fixes.patch".
>
> The patch fixes some bugs where the garbage collector (GC) will free an object
> that is being created and hasn't been put onto the Lua stack or added to a
> table yet. Also it protects the GC from recursive calls
> (allocator->GC->allocator->GC or GC->allocator->GC).
>
> Since this patch only fixes bug related to running the garbage collector from
> the allocator function, I only recommend using it if you need to restrict the
> amount of memory used by a Lua script. Also there maybe more bugs that I
> haven't found yet, so use at your own risk.
>
> Also attached is a new version of lua_memlimit.c + some lua scripts for stress
> testing. The new version adds a lot of checks and debug messages to the
> allocator for debugging GC bugs. There are some #define's at the top to
> control how many debug messages are enabled and what checks to run. Does
> anyone know a good set of Lua scripts for testing all Lua features? Do the
> scripts that come with lua 5.1.3 test all the language features?
>
> Summery of bug that the patch fixes (warning long read):
> Function "lua_setfield" in lapi.c
> This function created a Lua string and stored it in a variable on the C
> stack, had to move the string to the Lua stack so the GC wouldn't free the
> string before the key/value was added to the table. This is something that
> can effect other C libraries that create temporary Lua objects on the C
> stack. If another allocation happens before the object is put on the Lua
> stack or added to a table the GC might free the object.
>
> Function "lua_gc" in lapi.c
> Added protection from recursive LUA_GCCOLLECT calls. Also LUA_GCSTOP &
> LUA_GCRESTART can be used to stop a full GC.
>
> Function "f_parser" in ldo.c
> I haven't look to deeply into the Lua parser yet, but it looks like there
> are a lot of places where objects are created and not placed somewhere that
> the GC can find. So for now I just stop the GC before the parser runs, then
> restart it after.
>
> Function "checkSizes" in lgc.c
> This function was shrinking a temp. buffer used to concatenate strings. Now
> it will not shrink the size below the in use size needed by the string
> concat. function.
>
> Function "luaE_newthread" in lstate.c
> The new lua_State object wasn't fully allocated before a link to it was
> added to the GC. I only had to move the call to "luaC_link" to the end of
> the function.
>
> Function "luaH_new" in ltable.c
> The table object wasn't fully allocated before a link to it was added to the
> GC. I only had to move the call to "luaC_link" to the end of the function.
>
> Function "newlstr" in lstring.c
> This function creates a new Lua string object and adds it to the global
> string hashtable, it also calls "luaS_resize" to expand the string hashtable
> when it gets too crowded. The change needed here was to delay adding the
> string to the hashtable until after the hashtable was resized.
>
> Function "luaV_concat" in lvm.c
> Same as last patch, the temp. buffer has it's "in use" size updated so the
> GC will not shrink the buffer below the "in use" size. The "in use" size is
> reset to 0 after the buffer's contents are copied to a new Lua string.
>
> --
> Robert G. Jakabosky
>
