scope-aware values

[John Belmonte <john@...>]

I mentioned this briefly in my workshop presentation on exceptions, but
I encourage those interested to read the short page on exception safety
for the D Programming Language [1].  It explains why neither the
try-finally nor "resource acquisition is initialization" (RAII) idioms
for cleaning up resources in the face of exceptions yield readable, easy
to maintain code.  As a better solution, the D language has native
support for cleanup code which runs at the end of the scope (either on
success, failure by exception, or both) yet may be placed anywhere
within the block.  The C++ scope guard pattern [2] offers similar
functionality.

Let's suppose we had a variable class in Lua called "scoped".  It
behaves just as "local" except that when the variable goes out of scope
its value is called.  Obviously the value must be a function or an
object implementing the call metamethod.  The call has one argument,
which is an exception value if the scope exited by exception, else nil.
Synopsis:

  function add5(x)
      scoped bar = function(e) print('** scope exit', e) end
      return x + 5
  end

  > =add5(5)
  ** scope exit      nil
  10
  > =add5('foo')
  ** scope exit      attempt to perform arithmetic on local 'x' [...]
  stdin:3: attempt to perform arithmetic on local 'x' (a string value)
  stack traceback: [...]

As far as implementation, I believe the upvalue system can be employed
as Rici did for his non-local-exits [3].

What interesting things could we do with this?  Here's a solution to
Rici's example problem of a utility function which calls another
function with redirected output:

  function with_output(f, func, ...)
      local out = io.output()
      scoped function finally() io.output(out) end
      io.output(f)
      return func(...)
  end

Consider also a scope utility class which manages functions to be called
at scope exit.  It lets you queue functions to be invoked on scope
success, failure, or both.  Using such a utility, here is Rici's problem
again:

  function with_output(f, func, ...)
      scoped sm = ScopeManager()
      local out = io.output()
      sm.on_exit(function() io.output(out) end)
      io.output(f)
      return func(...)
  end

(I'm assuming ScopeManager is implemented with closures, hence the dot
notation for method calls.)  That's more verbose, but the usefulness of
the scope manager becomes clear in more complicated situations.  Here is
the email message sending problem from D's page on exception safety:

  function send_mail(msg)
      scoped sm = ScopeManager()
      do
          scoped sm2 = ScopeManager()
          var orig_title = msg.Title()
          msg.SetTitle("[Sending] " .. orig_title)
          sm2.on_failure(function() msg.SetTitle(orig_title) end)
          copy_msg(msg, SENT_FOLDER)
      end
      sm.on_success(function() set_msg_title(msg.ID(), SENT_FOLDER,
          msg.Title()) end)
      sm.on_failure(function() remove_msg(msg.ID(), SEND_FOLDER) end)
      smtp_send(msg) -- do the least reliable part last
  end

The advantages of this compared to try-finally and RAII are:  1) code to
cleanup a resource is placed in proximity to acquiring the resource; 2)
writing of extraneous cleanup classes is avoided; and 3) excessive
nesting is avoided.

So could this be used as a try-except construct also?  Let's try adding
the requirement that the scope-aware value must re-raise any error,
otherwise it will be suppressed.  Now we can write a try-except as follows:

  do
      scoped function catch(e)
          -- Except block.  E.g.:
          --   Use e for conditional catch
          --   Re-raise with error(e)
      end
      -- Try block
      --
  end

It may seem strange to have the catch block in front of the try block.
I don't mind it much because this makes it clear that variables in the
try block are not accessible during the catch.  Nonetheless it could be
cleaned up with some token filtering or metalua.  Here is an example of
committing changes to an object database, where we retry a certain
number of times if there is a conflict:

  function Database:commit()
      for attempt = 1, MAX_TRIES do
          scoped function catch(e)
              if instance_of(e, DatabaseConflictError) then
                  if attempt < MAX_TRIES then
                      log('Database conflict (attempt '..attempt..')')
                  else
                      error('Commit failed after '..attempt..' tries.')
                  end
              else
                  error(e) -- NOTE: no-op if e is nil, see aside below
              end
          end
          self.commit()
          return
      end
  end

[Aside: the Lua error() function behaves as a no-op when called with
nil.  This contradicts the Lua manual, but is a useful characteristic
which I'd like to see formalized.]

We've now introduced some ambiguity though.  Consider when there are
multiple "scoped" variables within a given scope, each of which are
supposed to be called on exit.  What if two or more raise exceptions?
Should this be a fatal error?  If not, which exception do we finally
propagate?  I'd rather avoid this mess by adding the constraint that
there be only one "scoped" variable per scope.  Applications can decide
how to resolve these situations via their scope manager.

Here is an alternative to adding the "scoped" variable class.  Say we
have a "with..as" statement similar to Python [4].  This is not the
Pascal "with" which opens up an objects members to the local scope,
which is apparently unworkable in Lua [5].  Rather, it's a way to define
a single value which manages a new scope, along with a variable for
accessing that value.  Like Python, binding to an explicit variable (the
"as" part) is optional.  So for example:

  function with_output(f, func, ...)
      with ScopeManager() as sm do
          local out = io.output()
          sm.on_exit(function() io.output(out) end)
          io.output(f)
          return func(...)
      end
  end

and here is a try-except:

  with function (e)
      -- Except block.  E.g.:
      --   Use e for conditional catch
      --   Re-raise with error(e)
  end do
      -- Try block
      --
  end

I think "with" could be employed for other interesting things by way of
some extra metamethods.  Instead of overloading the call metamethod on
scope exit, we could add an "exitscope" metamethod.  Maybe an
"enterscope" would be useful too.  And although opening up the objects
fields doesn't work for writes, I think it would still be useful for
reads.  Then with our ScopeManager we could avoid the "sm" variable
altogether:

  function with_output(f, func, ...)
      with ScopeManager() do
          local out = io.output()
          on_exit(function() io.output(out) end)
          io.output(f)
          return func(...)
      end
  end


Just some ideas to toss around-- I should really be writing my Gem.

--John



 [1] http://www.digitalmars.com/d/exception-safe.html
 [2] http://www.ddj.com/dept/cpp/184403758
 [3] http://lua-users.org/lists/lua-l/2005-08/msg00357.html
 [4] http://www.python.org/dev/peps/pep-0343/
 [5] http://lua-users.org/lists/lua-l/1999-07/msg00037.html