Re: Re[2]: state of the Lua nation on resource cleanup

[Matthew Wild <mwild1@...>]

On Sat, May 30, 2009 at 5:52 PM, Cosmin Apreutesei
<cosmin.apreutesei@gmail.com> wrote:
> On Sat, May 30, 2009 at 18:43, David Manura <dm.lua@math2.org> wrote:
>> On Sat, May 30, 2009 at 4:19 AM, Cosmin Apreutesei wrote:
>>>> That being said, I'd dare making a proposal that I've been giving some
>>>> thought, maybe I can hear your opinions about it (well, aside from the
>>>> trouble it would mean to implement it): how about an __immediate_gc
>>>> metamethod that would ensure finalization of a table or userdata when
>>>> the last reference goes out of scope.
>>
>> On Sat, May 30, 2009 at 4:51 AM, Bulat Ziganshin wrote:
>>> you can't work this way with external resources. imagine for example
>>> that your program opens files and process it some way. if file will be
>>> not closed until gc/exit, user trying to remove this file from other
>>> program will got unexpected fail
>>
>> Cosmin's proposal though is to extend Lua to support both tracing
>> garbage collection and deterministic destruction, with the latter done
>> through reference counting.  The choice between the two would be
>> selectable per object, depending on the metamethod defined on the
>> object.
>>
>> Reference counting is one way to achieve deterministic release of
>> resources.  However, it is not the only solution [1].  Perl supports
>> the reference counting approach.  Some implementations of Python
>> support it too (C not Java [2-3]), but it is an implementation
>> decision, and the more portable method is to use the with statement
>> [4], which is similar to the proposals made for Lua, where resources
>> are released simply upon scope exit, irregardless of reference counts
>> (which in a correct program normally should be zero upon scope exit).
>>
>>>> But that breaks garbage-collecting semantics, which assures me never
>>>> to have invalid pointers no matter what... IMHO this brings me back
>>>> down to the C-level of responsibility (watching the stack).
>>
>> Even reference counts have their limitations (e.g. cycles [3]), and
>> you are not prevented from doing dumb things like setting the variable
>> to nil and then trying to access the variable or accidentally storing
>> the resource in a global variable, which prevents the resource from
>> being released upon scope exit as might be assumed happens (thereby
>> only hiding the error).
>
> I think the problem with ref-counting is the performance hit of the
> cycle detection algorithm (although it's usually small potatoes
> compared to the finalization code itself -- as it was noted before,
> even triggering collectgarbage() usually takes less) -- cycles may not
> be such a big problem in practice, but if you go down that path, you
> gotta have full correctness. There might be better ways to implement
> determnism in gc, just don't know of any myself.
>
> I also agree that hanging references could be a problem too, though I
> assume it's easy to check with a small checking script, and you could
> always impose yourself more discipline :D
>
> I'd like to express just 2 more points and then I'm kicking myself out:
>
> 1) I'd rather not have the distinction between a
> closed/invalid/useless handler and a nil binding (that is, no
> binding), the same way lua assures me I'll never have invalid pointers
> thus no memory trashing or access violations ever.
>

What you are missing is that 'with' is simply an extension of what we
do already. You seem to be referring to your:

with f = io.open("file.txt") do
    somefile = f
end

It is true that a somefile:read() will fail. But the code above is
exactly the same as this:

f = io.open("file.txt")
somefile = f
f:close();

Trying to use somefile in either case will raise "attempt to use a
closed file" error.

> 2) the with/scoped approach has IMHO too narrow application. It is
> only good when the flow control of the program goes in sync with
> resource allocation/finalization (acquire -> use -> dispose). But
> think about a Win32 binding, with tons of handlers of many types
> floating around in tables, being passed around from function to
> function... Who should take care of their destruction? At this point
> you've got to implement a resource manager so you can have authority
> over finalization, since you're doing it all by yourself -- gc sittin'
> there doing nothing for you -- just like java :)
>
> Instead, I don't want to manage resources and create policies about
> their use. It's not that I'm lazy, but the bookkeeping really adds no
> value to my programs. All I want is to say t = nil, where t is a
> spaghetti tree of object handlers intermixed with memory objects
> (colors, canvases, panels, ...strings, ...) and be sure they're gone:
> no delay, no collectgarbage(), no invalid pointers on scope exit. Out
> and clear.
>

That's fine! You can still do that :)

The problem is that sometimes you /don't/ want to do that.

Do you never use file:close()? I do. The point is that sometimes you
want to be certain that file:close() is called as soon as you leave a
code block, not at some unspecified point in the future.

Files are an easy target, Lua already has them built in. But imagine
multi-threading, mutexes, and other things you really can't wait to be
freed.

Matthew