Re: block-scope finalization

[Coda Highland <chighland@...>]

On Mon, Nov 23, 2015 at 9:07 AM, Viacheslav Usov <via.usov@gmail.com> wrote:
> I have thought more about my original proposal, which was to introduce a new
> keyword that makes a variable subject to deterministic finalization.
>
> The chief difficulty is that an object referenced by such a variable can be
> assigned to another variable (stored in a table, captured in a closure), and
> vice versa. And, if that is not prevented, then the other variable may end
> up holding a reference to a finalized object. Today's resurrection could
> also result in that, but today resurrection is fully controlled by the
> finalizer and hence its library's writer; changing that will make a lot of
> existing libraries invalid.
>
> At this point it is tempting to say "let's make it impossible to assign
> block-scope vars to other vars, etc"; after a minute's reflection it is
> clear that such block-scope variables would be useless. So that approach
> won't work. We should not finalize anything while it is still being
> referenced.
>
> As far as I can tell, there is only one well-known method for deterministic
> finalization: reference counting. It is also well known that it does not
> work when references can be circular. However, a combination of reference
> counting AND garbage collection will ensure that finalization WILL happen,
> and will happen deterministically if a certain coding paradigm is followed,
> such as only storing reference-counted objects in local vars. The
> deterministic condition can probably be relaxed to cover a larger set of use
> cases, but local-vars-only would already be pretty good.
>
> I realize that ref counting will introduce certain overhead; but before we
> even go there, is there any fundamental reason why ref counting + GC cannot
> work in Lua? Note that such hybrid solutions exist in other languages, such
> as Python [1].
>
> Cheers,
> V.
>
> [1] https://docs.python.org/2/extending/extending.html#reference-counts
>

I think that we're actually looking at an XY issue here.

Deterministic finalization is a means to a particular end, but what is
that desired end? Ultimately: We want a specific function to be called
when control exits a block, no matter how it exits. In C++, we do this
using block scoping on a local variable. In Python, we have "finally:"
blocks and we have context managers and the "with" keyword; Java has
finally and try-with-parameters for the same.

In Lua, we COULD achieve this with some fairly ugly pcall hackery:

function finally(method, after)
  var result = table.pack(method)
  -- Note: Like in C#/Java/Python, an exception in the finally block
  -- results in the rest of the block being skipped and any previous
  -- exception to be lost.
  after()
  if result[1] then
    return table.unpack(result, 2, result.n)
  else
    error(result[2], 0)
  end
end

finally(function()
  -- do stuff here
end, function()
  -- finalize here
)

You could even create a context handler:

function with(...)
  var args = table.pack(...)
  var method = args[args.n]
  finally(method(table.unpack(args, 1, args.n-1)),
    function()
      for i in 1, args.n-1 do
        args[i]:close()
      end
    end
  )
end

with(io.open("filename.txt", "r"), function(f)
  -- do stuff with f
end)

Like in Python or Java, if you hold on to a reference to the context
handler object, it's still a valid object, but it'll have already had
its close() method called.

I... actually rather like this with() function I've written here. I
was GOING to use this code as an example of where we could try to
improve on things, but in the end I wrote something that actually
looks pretty darn nice. (It might be somewhat more efficient to do
this using the C API due to the nature of varargs stack manipulation,
but it does WORK in Lua.)

That said, I haven't actually TESTED any of the above code, and it
wouldn't even work in Lua 5.1 without some modification (which is what
most of my code targets due to LuaJIT), but I think the concept is
pretty solid, and I do like the varargs tricks making it pretty
flexible.

/s/ Adam