[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
- Subject: Re: PATCH: fixes bug with calling garbage collector from custom lua_Alloc
- From: "Bogdan Marinescu" <bogdan.marinescu@...>
- Date: Sun, 4 May 2008 15:57:43 +0300
By the way, when I apply your last patch (emergency_gc_fixes.patch) to
an original version of Lua 5.1.3, this is what I get (note the
warnings about offsets in 'lapi.c'):
patching file src/lapi.c
Hunk #1 succeeded at 657 (offset 1 line).
Hunk #2 succeeded at 912 (offset 1 line).
patching file src/ldo.c
patching file src/lgc.c
patching file src/lstate.c
patching file src/lstring.c
patching file src/ltable.c
patching file src/lvm.c
Could you check if the patch is OK?
On Sun, May 4, 2008 at 3:16 PM, Bogdan Marinescu
<bogdan.marinescu@gmail.com> wrote:
> I applied your new patch. Running the garbage collector every time a
> new allocation is needed clearly decreases the required memory amount,
> but also slows the script a lot. An example on 'life.lua' with TLSF
> (now it runs with TLSF; I think it was a problem with my statistics
> module, not with your patch):
>
> TLSF without GC being called by the allocation function: needs 182,000
> bytes (this is the reference).
> TLSF with GC running before each new allocation: needs 67,000 bytes of
> RAM (an excellent value, but very slow)
> TLSF with GC running only after a new allocation couldn't be
> satisfied: needs 180,000 byes of RAM (not much of an improvement)
> TLSF with GC being called when a limit of 64000 bytes is reached:
> needs 72000 bytes (again an excellent value, and the performance seems
> very good).
>
> This is only one test, so it's hard to extrapolate, but the results
> seem interesting, and I can already presume that running the GC all
> the time will not have much practical value, since it slows Lua a lot.
> Also, the 64000 limit on the last case presented above is quite
> interesting, because once you move to other values (upper or lower)
> you'll get a 'not enough memory' error. I was only able to run the
> script in 72000 bytes with the 'magical' 64000 limit, and not anything
> else. I can't explain this yet. Also, when I try to run
> 'factorial.lua' with the 'always call GC' method (TLSF limited to 39k
> of RAM) I get a segfault with this stack trace:
>
> #0 0x00000000 in ?? ()
> #1 0x080504a6 in luaD_precall (L=0x80755a4, func=0x807581c,
> nresults=-1) at lua-5.1.3/src/ldo.c:319
> #2 0x0805e86c in luaV_execute (L=0x80755a4, nexeccalls=2) at
> lua-5.1.3/src/lvm.c:610
> #3 0x08050706 in luaD_call (L=0x80755a4, func=0x80757f8, nResults=-1)
> at lua-5.1.3/src/ldo.c:377
> #4 0x0804b33f in f_call (L=0x80755a4, ud=0xfff54154) at
> lua-5.1.3/src/lapi.c:801
> #5 0x0804f8a9 in luaD_rawrunprotected (L=0x80755a4, f=0x804b315
> <f_call>, ud=0xfff54154) at lua-5.1.3/src/ldo.c:116
> #6 0x08050a62 in luaD_pcall (L=0x80755a4, func=0x804b315 <f_call>,
> u=0xfff54154, old_top=48, ef=36) at lua-5.1.3/src/ldo.c:463
> #7 0x0804b3d3 in lua_pcall (L=0x80755a4, nargs=0, nresults=-1,
> errfunc=2) at lua-5.1.3/src/lapi.c:822
> #8 0x0806b4f9 in docall (L=0x80755a4, narg=0, clear=0) at
> lua-5.1.3/src/lua.c:102
> #9 0x0806bd56 in handle_script (L=0x80755a4, argv=0xfff54514, n=1) at
> lua-5.1.3/src/lua.c:250
> #10 0x0806c225 in pmain (L=0x80755a4) at lua-5.1.3/src/lua.c:362
> #11 0x080504a6 in luaD_precall (L=0x80755a4, func=0x80757d4,
> nresults=0) at lua-5.1.3/src/ldo.c:319
> #12 0x080506ef in luaD_call (L=0x80755a4, func=0x80757d4, nResults=0)
> at lua-5.1.3/src/ldo.c:376
> #13 0x0804b4b6 in f_Ccall (L=0x80755a4, ud=0xfff543cc) at
> lua-5.1.3/src/lapi.c:847
> #14 0x0804f8a9 in luaD_rawrunprotected (L=0x80755a4, f=0x804b403
> <f_Ccall>, ud=0xfff543cc) at lua-5.1.3/src/ldo.c:116
> #15 0x08050a62 in luaD_pcall (L=0x80755a4, func=0x804b403 <f_Ccall>,
> u=0xfff543cc, old_top=12, ef=0) at lua-5.1.3/src/ldo.c:463
> #16 0x0804b504 in lua_cpcall (L=0x80755a4, func=0x806c0a1 <pmain>,
> ud=0xfff54400) at lua-5.1.3/src/lapi.c:857
> #17 0x0806c315 in lua_main (argc=2, argv=0xfff54514) at lua-5.1.3/src/lua.c:396
> #18 0x0806d1b3 in main (argc=6, argv=0xfff54504) at luastat.c:493
>
> (I applied your latest patches). I still don't know if the error is in
> my code or in your code; however, I only modified 'lua.c' very briefly
> for my statistics module, and I find it hard to believe that my
> modification could lead to a stack trace like the one above :)
> Also, in your new 'memlimit.c' file, I noticed that you don't use the
> 'collecting' guard variable anymore. Is this an omission, or it's a
> consequence of the patch?
>
>
>
> On Sun, May 4, 2008 at 6:18 AM, Robert G. Jakabosky
> <bobby@sharedrealm.com> wrote:
> > Updated patch attached "emergency_gc_fixes.patch".
> >
> > The patch fixes some bugs where the garbage collector (GC) will free an object
> > that is being created and hasn't been put onto the Lua stack or added to a
> > table yet. Also it protects the GC from recursive calls
> > (allocator->GC->allocator->GC or GC->allocator->GC).
> >
> > Since this patch only fixes bug related to running the garbage collector from
> > the allocator function, I only recommend using it if you need to restrict the
> > amount of memory used by a Lua script. Also there maybe more bugs that I
> > haven't found yet, so use at your own risk.
> >
> > Also attached is a new version of lua_memlimit.c + some lua scripts for stress
> > testing. The new version adds a lot of checks and debug messages to the
> > allocator for debugging GC bugs. There are some #define's at the top to
> > control how many debug messages are enabled and what checks to run. Does
> > anyone know a good set of Lua scripts for testing all Lua features? Do the
> > scripts that come with lua 5.1.3 test all the language features?
> >
> > Summery of bug that the patch fixes (warning long read):
> > Function "lua_setfield" in lapi.c
> > This function created a Lua string and stored it in a variable on the C
> > stack, had to move the string to the Lua stack so the GC wouldn't free the
> > string before the key/value was added to the table. This is something that
> > can effect other C libraries that create temporary Lua objects on the C
> > stack. If another allocation happens before the object is put on the Lua
> > stack or added to a table the GC might free the object.
> >
> > Function "lua_gc" in lapi.c
> > Added protection from recursive LUA_GCCOLLECT calls. Also LUA_GCSTOP &
> > LUA_GCRESTART can be used to stop a full GC.
> >
> > Function "f_parser" in ldo.c
> > I haven't look to deeply into the Lua parser yet, but it looks like there
> > are a lot of places where objects are created and not placed somewhere that
> > the GC can find. So for now I just stop the GC before the parser runs, then
> > restart it after.
> >
> > Function "checkSizes" in lgc.c
> > This function was shrinking a temp. buffer used to concatenate strings. Now
> > it will not shrink the size below the in use size needed by the string
> > concat. function.
> >
> > Function "luaE_newthread" in lstate.c
> > The new lua_State object wasn't fully allocated before a link to it was
> > added to the GC. I only had to move the call to "luaC_link" to the end of
> > the function.
> >
> > Function "luaH_new" in ltable.c
> > The table object wasn't fully allocated before a link to it was added to the
> > GC. I only had to move the call to "luaC_link" to the end of the function.
> >
> > Function "newlstr" in lstring.c
> > This function creates a new Lua string object and adds it to the global
> > string hashtable, it also calls "luaS_resize" to expand the string hashtable
> > when it gets too crowded. The change needed here was to delay adding the
> > string to the hashtable until after the hashtable was resized.
> >
> > Function "luaV_concat" in lvm.c
> > Same as last patch, the temp. buffer has it's "in use" size updated so the
> > GC will not shrink the buffer below the "in use" size. The "in use" size is
> > reset to 0 after the buffer's contents are copied to a new Lua string.
> >
> > --
> > Robert G. Jakabosky
> >
>
