Re: Garbage collection durint pcall.

[Francisco Olarte <folarte@...>]

Sean:

On Sun, Jun 2, 2019 at 12:06 PM Sean Conner <sean@conman.org> wrote:
> It was thus said that the Great Francisco Olarte once stated:
> > I'm trying (unsuccessfully) to see in the docs if a function argument
> > can be collected during a pcall. What I mean is:
> > 0.- I push a function.
> > 1.- I push a userdata object as argument.
> > 2.- I do lua_pcall(L,1,1,0) ( Or lua_call or other similar stuff ).
> > Can my object be garbage collected before returning from pcall (i.e.,
> > if the function sets its argument to nil and then calls collectgarbage
> > ).?
> > I need to know this because the object I push has a __gc method which
> > destroys it, and I need to access some data inside it just after
> > return.
> > ( I know I can keep it alive by storing it in the registry or dupping
> > it or anything similar ).
>   I've gone back up to the first message basically because I don't know
> where else to insert this into the chain of emails.  So, right back at the
> start.
>
>   I'm looking at the Lua manual for lua_pcall(), and past the function
> protocol, all the way to the right, is this notation:
>
>         [-(nargs + 1),+(nresults|1),-]

That I did.

> This tells me that all the parameters, plus the function itself, is popped
> from the Lua stack, and any results from the called function are pushed onto
> the stack (or an error---this is lua_pcall() after all).  Since you create a
> userdata and pass it right to the function, once that function returns,
> *even if it didn't explicitely nil out that parameter* is it no longer
> accessible to Lua and thus, subject to garbage collection.

I did that, but my ( as I've said it, erroneous asumption ) was this
is just what happened. I thought object was available for GC "once
that function returns". The thing being, once that function returns
I'm in my C code, so lua CAN NOT collect ( My C code has the control )
so I could just "look a bit" inside the userdata. I.e., I assumed the
object could be dead and collectable, but not still collected. A
later, simple pure-lua experiment proved me wrong, it seems lua pops
my parameters and puts it somewhere in some kind of locals ( i.e., it
works similar as if the parameters where magically initialized locals
) ( makes sense from some points of view, and it does not matter how
it exactly does it, the point is the referece in my C-module lua stack
is gone and the things can be collected DURING the function.

>   The same applies to lua_call(), lua_callk() and lua_pcallk()---the calling
> parameters are popped off the stack during the process of the call.

That I know, the problem is I thought lua did it in another way.

>   That you may still have access to it via a pointer from lua_newuserdata()
> is beside the point---how you constructed the function call means that once
> said call is done, that particular piece of datum is no longer accessible
> according to the Lua documentation.

I know it is no longer accesible AFTER the pcall, I wanted to know if
it could be COLLECTED. I needed it uncollected, it did not matter to
me whether it was ACCESSIBLE. I fixed the problem by making it
accessible.

  It's somewhat similar to this bit of C
> code:
>
>         int *p = malloc(sizeof(int));
>         if (p)
>         {
>           free(p);
>           *p = 3;
>         }
>   It's a violation of the C standard but it will still probably work (only
> the issue may be spelled out a bit better in the C standard).

It's not exactly the same, I wouldn't have problems if it where plain
C. C is deterministic and does not need to run the garbage collector
to do the free. This code is more or less equivalent to creating the
userdata, doing the pcall, calling collect_garbage(collect), and
accessing the pointer, I was not doing ( and knew no to do ) that.

In Lua newuse4rdata does the malloc, but the garbage collector does
the free. So someone has to call the garbage collector after
everything is done. I'm not going to try it, but imagine, in my C
module, I do a lua_newuserdata(), and store the pointer to it. Then I
do lua_pop(). Then I access the memory. (I AM NOT DOING THAT, this is
for discussion ). After the lua_pop the UD is collectable, but the GC
has got no oportunity to run, lua_pop() does not free the object, just
makes it inaccessible from lua. This is not problem with lua-pop, as I
have no reason to do pop-read instead of read-pop, but I had reasons
to do pcall-read instead of read-pcall.

What I was ( erroneously, but could have gotten away for it for a long
time until some thing increases memory pressure and I get a core ).
Was something like this, create the userdata, store the ptr, pcall,
and just after pcall, without any intervening code, read a flag.
Something like

push function.
my_obj * mo = new(lua_new_userdata(L, sizeof(my_obj))) my_obj(creation
parameters);
... some more stuff.
lua_pcall();
// NOTHING ELSE DONE BETWEEN THESE, GC cannot run AFTER pcall, CPU is mine.
bool my_flag = my_obj->flag; // BAD, GC may have run and collected INSIDE Pcall.

I knew the UD was dead, but was assuming it could not be collected
until I call another lua api ( I assumed even lua_pushnil could have
triggered a gc cycle, the access was literally the next C line after
the pcall ).

I noticed it, I analyzed it and knew I did it because coming from
punch-cards, 16k ram and 1Mb hdd I tend to be a little spartan in ram
usage until I realize icons use 1Mb now. I fixed it by pushing a copy,
which was the right thing to do, but I wanted to know if args where
live until return or not. My experiment revealed that to not be true (
I did in pure lua, but I assumed if a table can be collected a
userdata can be too ).

Francisco Olarte.