Re: Let deprecate to-be-close variables as harmful concept

I've roughly seen four concepts for resource management:

(0. No language support, do it yourself. Error-prone; often only code for the "happy path" is written (so errors that don't terminate the application result in resource leaks).)
1. "with" blocks. In Python this is a statement with a block scope. In languages like Lua, this can be implemented using closures (for example, in Lua 5.1 I use a with_open akin to Scheme's with-(input|output)-(from|to)-file).

2. "defer" statements, as found e.g. in Go or Zig, which run in reverse order when the function (Go) or scope (Zig) is left (normally or with an error).

3. SBRM (Scope-Bound Resource Management) like Lua's to-be-closed variables: Resources are released when the corresponding variables go out of scope.

4. Reference counting, which ensures that resources are released when they aren't needed any more, no matter how contrived the program logic. Works as long as there are no reference cycles among resources (which there practically almost never are). The special case of 0-1-reference counting can to a large extent be done by "ownership" models like Rust's, but those are very much out of scope for Lua.

I think 3. is the best option for Lua here:

1. Becomes awkward as the more resources you use, the deeper your nesting goes. The closure-based approach has the shortcomings of function usage (for example not allowing break).
2. This is a pretty decent solution; it is slightly more verbose however (another statement). I also don't like tying this to block- or function scope; this means in practice resources will live longer than they have to if the programmer (say, the function/block does some expensive computation after being done with the resource; the resource will only be released after that). It has the small advantage vs. to-be-closed variables that you need less boilerplate if you have cleanup code that naturally isn't tied to an object.
3. The best solution for Lua, IMO. Very concise - just add "<close>" after the resource variable name. Definitely much better than the error-prone "local f = io.open(...); [...]; f:close()" antipattern, which won't clean up immediately if the "[...]" throws an error that is caught later on. Let's the compiler figure out for you when you last use a variable (and when it is thus safe to clean it up).
4. Not really an option for Lua, since Lua doesn't base its garbage collection on reference counting; not using RC is presumably also better for performance, and is simpler than having to use something like a RC + tracing/generational GC approach to collect reference cycles.

There's also the case of iterators - like io.lines - having to do cleanup. Prior to 5.4's introduction of the to-be-closed iterator variable for the generic for loop, pretty much the best solution for this was to pass a closure, with all the downsides this has (for example, inside a closure, you can't return from the enclosing function, and you can't break an enclosing loop; also comes at a performance cost).

The only main downside of to-be-closed variables I see at the moment is that it interferes with tail calls. I think the simplest solution would be to always close all to-be-closed variables before tail calling, preserving the simple rule that "return f(...)" is always a tail call.

In principle, reference counting is the cleanest solution in my opinion, but it would come at a performance cost and much more complexity than to-be-closed variables.

I don't really see how "[t]his implementation creates more problems than [it] solve[s]". Some complexity is necessary; programmers need to think about resource management.
It solves the problem of simple resource management pretty cleanly. You can now properly work with a file and all you have to do for that is add <close> after a variable name.
In more complex scenarios, programmers might of course introduce bugs. But I would argue that they probably
(1) introduced even more bugs before this feature existed, due to forgetting to release resources (esp. on error!), and
(2) if they wrote their code very carefully to handle resource management, they just don't need to use to-be-closed variables;
they can stick with whatever pure Lua constructs they have developed.

- Lars

On 03/10/2023 09:27, kovserg33@gmail.com wrote: