Re: state of the Lua nation on resource cleanup

[David Manura <dm.lua@...>]

On Tue, Feb 10, 2009 at 6:07 PM, Roberto Ierusalimschy wrote:
> In all cases the key point is "on exceptions".

"On exceptions" is possibly the most key point, but "on scope exit" is
also important, even on the absence of exceptions.

On Fri, Feb 13, 2009 at 7:14 AM, M Joonas Pihlaja wrote:
> The main reason I'm not a huge fan of this solution is that it doesn't
> deal with the far more common non-exceptional case, leaving it up to
> the user to call finalizers explicitly.

On Fri, Feb 13, 2009 at 9:03 AM, John Belmonte wrote:
> ...I think this solution overlooks the usefulness of
> having a cleanup mechanism that covers all exit cases.

Here's an actual function I was maintaining today.  This was in C++,
but it could just as well have been in Lua (e.g. using one of
http://lua-users.org/wiki/GraphicalUserInterfaceToolkits ).  The code
could have very nicely been written in the following manner if Lua
allowed it:

  function render(object)
    cursor_off()
    finally cursor_on() do
      local temp_palette = current_palette
      finally
        current_palette = temp_palette
        draw(object)
      do
        status_user("printing...")
        finally status_user("printing done") do
          render_helper(object)
        end
      end
    end
  end

That is, it uses a "try / finally" like syntax but with the try and
finally reversed: "finally <block> do <block> end".

Here's what a straightforward translation of that would look like
under Roberto's recent "pcall-with-collectgarbage" proposal:

  function render(object)
    cursor_off(); newresource(cursor_on)
    local temp_palette = current_palette
    do
      local screen = newresource(function()
        current_palette = temp_palette;
        draw(object)
      end)
      status_user("printing...")
      do
        local s = newresource(function() status_user("printing done") end)
        render_helper(object)
        s:close()
      end
      screen:close()
    end
    cursor_on()
  end

Or we may prefer expressing it in a non-nested style like this:

  function render(object)
    -- setup (and teardown on exception)
    cursor_off(); newresource(cursor_on)
    local temp_palette = current_palette
    local screen = newresource(function()
      current_palette = temp_palette;
      draw(object)
    end)
    status_user("printing...")
    local s = newresource(function() status_user("printing done") end)
    render_helper(object)

    -- teardown (on no exception)
    s:close()
    screen:close()
    cursor_on()
  end

It's doable, but it has more bookkeeping, lacks spatial locality of
setup/teardown, and has more temporary functions.  (Related note:
lambdas are verbose in Lua --
http://lua-users.org/wiki/ShortAnonymousFunctions .)  It gets worse if
there are breaks and returns.  You'd probably end up writing a helper
function "newscope" to rewrite the code like this:

  function render(object)
    local scope = newscope() -- a resource that contains other resources

    -- setup (and teardown on exception)
    cursor_off(); scope.on_exit(cursor_on) -- create resource and
append to scope
    local temp_palette = current_palette
    local screen = scope.on_exit(function()
      current_palette = temp_palette;
      draw(object)
    end)
    status_user("printing...")
    local s = scope.on_exit(function() status_user("printing done") end)
    render_helper(object)

    -- teardown (on no exception) (note: do this also before any break/return's)
    scope:close()
  end

and perhaps also refactor the execution/teardown pattern into the
scope constructor:

  function render(object)
    scope(function(s)
      cursor_off(); s.on_exit(cursor_on)
      local temp_palette = current_palette
      s.on_exit(function()
        current_palette = temp_palette
        draw(object)
      end)
      status_user("printing...")
      s.on_exit(function() status_user("printing done") end)
      render_helper(object)
    end)
  end

which, in fact, basically has the same syntax as John's Lua
Programming Gem #13 using an unpatched Lua 5.1 (!).  The only
difference is that here the "scope" function used does not itself do a
pcall.  Rather, on the non-exceptional exit condition, it simply runs
all the registered on_exit functions in reverse order.  On the
exceptional exit condition, we leave it to pcall-with-collectgarbage
to cleanup the scope object.

Still, it retains a few of the same limitations as in Gem #13, such as
you can't use a regular return or break inside the function block:

  function render(object)
    scope(function(s) .... return .... end)  -- opps, scope swallows the return.
  end

For completeness, here's what a nested form of an approach like Gem
#13 would look like:

  function render(object)
    cursor_off()
    finally(cursor_on, function()
      local temp_palette = current_palette
      finally(function()
                  current_palette = temp_palette
                  draw(object)
               end,
      function()
        status_user("printing...")
        finally(function() status_user("printing done") end, function()
          render_helper(object)
        end)
      end)
    end)
  end
  -- note: the helper function "finally" internally does a pcall

It is, however, more typical of Lisp than Lua for control structures
and blocks (which is what these are) to be implemented as functions
rather than as built-in language constructs.  The previous example
also suffers from this but on a much more limited scale.  I feel this
obfuscates the code a bit, not to mention has some run-time effects.

Using the "with" proposal advocated by John, depending which variant
you want to use, the "finally"-like clause is in the form of an
expression or object rather than a block as in my first code example
above.  This is only slightly more burdensome (and the burden could be
alleviated further with a simpler lambda syntax that suggests a block
semantics):

  function render(object)
    cursor_off()
    with cursor_on do
      local temp_palette = current_palette
      with function()
        current_palette = temp_palette
        draw(object)
      end do
        status_user("printing...")
        with function() status_user("printing done") end do
          render_helper(object)
        end
      end
    end
  end
  -- or the non-nested form of this.


I'll leave this post with an idea...  How about merging John's "with"
proposal and Roberto's "pcall-with-collectgarbage" proposal?  That is,
have "with" handle cleanup on non-exceptional scope exit, and have
pcall-with-collectgarbage handle cleanup on exceptional scope exit.
This "with" would probably be implemented entirely by the Lua compiler
(inserting cleanup calls before each break, return, or non-exceptional
scope exit).  I think it would involve no change to the Lua VM.
"with" would merely be a syntactic sugar.

  function render(object)
    with s = newscope() do  -- note: compiler shall translate to e.g.
"local s = newscope()"
      cursor_off(); s.on_exit(cursor_on)
      local temp_palette = current_palette
      s.on_exit(function()
        current_palette = temp_palette
        draw(object)
      end)
      status_user("printing...")
      s.on_exit(function() status_user("printing done") end)
      render_helper(object)

      -- note: the compiler shall insert an s:close() here.
      -- (It will also do so before any break/returns--but what about
tail calls?)
    end
  end