Re: block-scope finalization

2015-11-23 20:58 GMT+01:00 Philipp Janda <siffiejoe@gmx.net>:

First of all, thanks for your answers, Roberto.

Am 23.11.2015 um 18:25 schröbte Roberto Ierusalimschy:

The chief difficulty is that an object referenced by such a variable can be
assigned to another variable (stored in a table, captured in a closure),
and vice versa. And, if that is not prevented, then the other variable may
end up holding a reference to a finalized object. Today's resurrection
could also result in that, but today resurrection is fully controlled by
the finalizer and hence its library's writer; changing that will make a lot
of existing libraries invalid.

I think it is a good practice to allow objects to be "finalized"
multiple times (that is, to allow their __gc metamethod to be called
multiple times) and also to allow references to finalized objects with
sensible behavior.

First, if the library does not protect metatables, any code can get the
__gc metamethod and call it explicitly.

IMHO, protecting the metatable is a good idea anyway so that no-one removes the `__gc` metamethod and you run out of resources somewhere else.

Second, it is a good practice (well, I think it is) to provide an
explicit way to "close" an object that needs finalization.

The io library, for instance, does that.

It's easy for the io library, because it stores pointers. Downside is that the garbage collector isn't aware of the extra memory, and memory fragmentation could be higher since there are multiple allocations for each userdata now. For every non-pointer userdata you'd have to add an extra field to indicate the state of finalization.
But now every (meta-)method has to check that the userdata is still valid, and most Lua code that uses those (meta-)methods has to check again unless it is ok with a simple getter/setter throwing an error. Also there are cases where explicitly "closing" an object is unsafe, e.g. if another object holds a pointer to that object. You can ensure the correct `__gc` order by storing references in the appropriate uservalue tables, but invalidating dependent objects is harder -- especially if the dependencies might change at runtime. Concrete examples from the last three libraries I created bindings for are renderers and textures in libSDL, memory pools and any other APR object in the Apache Portable Runtime library, and Fl_Input_Choice and its Fl_Input and Fl_Menu_Button subwidgets in FLTK.

Maybe the hypothetical "block" variable should only call `__gc` if the metatable is not protected (hoping that the author has taken proper care). Alternatively, a new (meta-)method could be introduced.

-- Roberto

Philipp

I don't think that using an garbage collector finalizer is the right way to go with a mark and sweep collector (even if it is the only way I see right now).

I've encountered the problem with transactions now in Lua, PHP, Java and C++. It was always desired that a function is called when the execution leaves the scope - in order and deterministic. In Lua and Java it was a pain, while in C++ and PHP it would be fairly straight forward to use objects allocated on the stack that would deal with leaving the scope of interest and calling a function. In PHP it would work because of the reference counting strategy for object collection - showing an advantage over mark and sweep collectors: Object garbage collector finalizers getting called right away when an object is nulled (given that the object on the stack is the only reference).

Thinking about this issue a while (and thinking to start a discussion on the mailing list on this topic) I haven't really had a good idea; But I was thinking that having some bytecode scope guards would be an option. Something like

-----

local fp = assert(io.open("hello.txt"))

-- ...

finally

if fp ~= nil then fp:close()

end

-----

This would not deal with coroutine yielding though. Though I think that could be worked around by adding hooks to the yield / resume functions, like

-----

local fp = assert(io.open("hello.txt"))

local hander = coroutine.addHandler {

yield = function() fp:close() end;

resume = function() fp = assert(io.open("hello.txt")) end

}

...

finally

coroutine.removeHandler(handler)

end

-----

So I think having a finally block option would greatly improve the overall situation, however the OOP finalizer technique would still offer the advantage of being able to cope with resource deinitialization on the class definition level.

In the end, I think I'd be more happy if Lua had an (optional) reference counting based GC that would execute finalizers in a deterministic way.

Cheers,

Eike