Re: Re[2]: state of the Lua nation on resource cleanup

[Cosmin Apreutesei <cosmin.apreutesei@...>]

On Sat, May 30, 2009 at 20:14, Matthew Wild <mwild1@gmail.com> wrote:
> On Sat, May 30, 2009 at 5:52 PM, Cosmin Apreutesei
> <cosmin.apreutesei@gmail.com> wrote:
>> On Sat, May 30, 2009 at 18:43, David Manura <dm.lua@math2.org> wrote:
>>> On Sat, May 30, 2009 at 4:19 AM, Cosmin Apreutesei wrote:
>>>>> That being said, I'd dare making a proposal that I've been giving some
>>>>> thought, maybe I can hear your opinions about it (well, aside from the
>>>>> trouble it would mean to implement it): how about an __immediate_gc
>>>>> metamethod that would ensure finalization of a table or userdata when
>>>>> the last reference goes out of scope.
>>>
>>> On Sat, May 30, 2009 at 4:51 AM, Bulat Ziganshin wrote:
>>>> you can't work this way with external resources. imagine for example
>>>> that your program opens files and process it some way. if file will be
>>>> not closed until gc/exit, user trying to remove this file from other
>>>> program will got unexpected fail
>>>
>>> Cosmin's proposal though is to extend Lua to support both tracing
>>> garbage collection and deterministic destruction, with the latter done
>>> through reference counting.  The choice between the two would be
>>> selectable per object, depending on the metamethod defined on the
>>> object.
>>>
>>> Reference counting is one way to achieve deterministic release of
>>> resources.  However, it is not the only solution [1].  Perl supports
>>> the reference counting approach.  Some implementations of Python
>>> support it too (C not Java [2-3]), but it is an implementation
>>> decision, and the more portable method is to use the with statement
>>> [4], which is similar to the proposals made for Lua, where resources
>>> are released simply upon scope exit, irregardless of reference counts
>>> (which in a correct program normally should be zero upon scope exit).
>>>
>>>>> But that breaks garbage-collecting semantics, which assures me never
>>>>> to have invalid pointers no matter what... IMHO this brings me back
>>>>> down to the C-level of responsibility (watching the stack).
>>>
>>> Even reference counts have their limitations (e.g. cycles [3]), and
>>> you are not prevented from doing dumb things like setting the variable
>>> to nil and then trying to access the variable or accidentally storing
>>> the resource in a global variable, which prevents the resource from
>>> being released upon scope exit as might be assumed happens (thereby
>>> only hiding the error).
>>
>> I think the problem with ref-counting is the performance hit of the
>> cycle detection algorithm (although it's usually small potatoes
>> compared to the finalization code itself -- as it was noted before,
>> even triggering collectgarbage() usually takes less) -- cycles may not
>> be such a big problem in practice, but if you go down that path, you
>> gotta have full correctness. There might be better ways to implement
>> determnism in gc, just don't know of any myself.
>>
>> I also agree that hanging references could be a problem too, though I
>> assume it's easy to check with a small checking script, and you could
>> always impose yourself more discipline :D
>>
>> I'd like to express just 2 more points and then I'm kicking myself out:
>>
>> 1) I'd rather not have the distinction between a
>> closed/invalid/useless handler and a nil binding (that is, no
>> binding), the same way lua assures me I'll never have invalid pointers
>> thus no memory trashing or access violations ever.
>>
>
> What you are missing is that 'with' is simply an extension of what we
> do already. You seem to be referring to your:
>
> with f = io.open("file.txt") do
>    somefile = f
> end
>
> It is true that a somefile:read() will fail. But the code above is
> exactly the same as this:
>
> f = io.open("file.txt")
> somefile = f
> f:close();
>
> Trying to use somefile in either case will raise "attempt to use a
> closed file" error.
>

That's correct, but I don't know what am I missing here.

>> 2) the with/scoped approach has IMHO too narrow application. It is
>> only good when the flow control of the program goes in sync with
>> resource allocation/finalization (acquire -> use -> dispose). But
>> think about a Win32 binding, with tons of handlers of many types
>> floating around in tables, being passed around from function to
>> function... Who should take care of their destruction? At this point
>> you've got to implement a resource manager so you can have authority
>> over finalization, since you're doing it all by yourself -- gc sittin'
>> there doing nothing for you -- just like java :)
>>
>> Instead, I don't want to manage resources and create policies about
>> their use. It's not that I'm lazy, but the bookkeeping really adds no
>> value to my programs. All I want is to say t = nil, where t is a
>> spaghetti tree of object handlers intermixed with memory objects
>> (colors, canvases, panels, ...strings, ...) and be sure they're gone:
>> no delay, no collectgarbage(), no invalid pointers on scope exit. Out
>> and clear.
>>
>
> That's fine! You can still do that :)

I can?

>
> The problem is that sometimes you /don't/ want to do that.
>
> Do you never use file:close()? I do. The point is that sometimes you
> want to be certain that file:close() is called as soon as you leave a
> code block, not at some unspecified point in the future.
>
> Files are an easy target, Lua already has them built in. But imagine
> multi-threading, mutexes, and other things you really can't wait to be
> freed.
>

With immediately-collected resources, the functionality of f:close()
remains, as replaced by f = nil. I don't think any control is taken
away.