Re: Lua Exception Patch (was Re: Start of Lua rewrite in C++)

["Thomas Lefort" <thomas.lefort@...>]

This rewrite initiative is a good opportunity to revive old hacking
notes of mine, where I address the problem of making exceptions
traverse Lua. The "partial solution" is equivalent to your patch. The
"alternative" is implemented in most automatic wrappers. The "complete
solution" boils down to the RAII pattern, which is best applied in a
full C++ setting.

from the wiki at http://lua-users.org/wiki/LuaAndExceptionsHackingNotes

=== Current algorithm ===

As of version 5.1 built as C++.

== Data structures ==
The "long jump buffer" is a linked list of volatile status.
{{{
struct lua_longjmp {
  struct lua_longjmp *previous;
  int dummy;
  volatile int status;
};
}}}
Its role is to keep track of the protected call stack. We start with
an empty global buffer.

== Catching errors ==
{{{
// chain a new local buffer, status = 0
// global buffer now points to it
try {
  // call
}
catch (...) {
  // force local status to non 0 (e.g. -1)
}
// restore global buffer
// return local status
}}}

== Throwing errors ==
{{{
// if empty global buffer then panic
// store status in global buffer
// throw its address !
}}}

== Discussion ==
The current algorithm makes it possible to throw exceptions from the
other side of Lua, but they will never make it to the other end, for a
good reason.

If indeed they weren't caught at Lua level at all, catching them later
would unwind violently out of Lua's call stack, leaving us with an
inconsistent Lua state. This is what happened when I tweaked Lua 5.0.2
that way.

To ensure a consistent Lua state while still carrying the exception to
the other side, we would need to store a copy of it in a safe place,
unwind normally to toplevel, and finally throw the copy. However,
because catch matches types (Stroustrup 14.3), we lose type
information when we `catch (...)`, and with that the ability to copy
generic exceptions.

=== A partial solution ===

To implement the copy mechanism discussed above, we could narrow the
type range of alien exceptions to the standard `std::exception`.
If the programmer needs to catch other exceptions, he can wrap them
into `std::exception` subclasses.
The remaining exceptions would still have to be silently ignored.

This solution has the advantage of simplicity, and works well with
hierarchical exception classes (one wrapper per root class). It is
however limited to situations where both ends can agree on the
`std::exception` wrapping.

NOTE: this means wrapper generators need to be adapted to this scheme.

As a temporary solution, one may also narrow the exception space to a
set of custom exceptions, that is, manually at Lua build time.

TODO: check copy of `std::exception`

=== Alternative: extending tolua++ ===
The above solution is not really proper. If we use a free software
binding generator such as [http://www.codenix.com/~tolua/ tolua++], we
can extend it so as to protect Lua from exceptions thrown by the bound
C++ code. This has the advantage of leaving the Lua source base
uncluttered.

Looking back, it appears as an appropriate solution in our very case,
and it should also be enough for casual applications. However it
entails translation between error systems and thus redundancies.

If the Lua source became (or forked) exception-safe, we could enable
it to catch C++ exceptions from Lua, either explicitely or through a
binding to an eventual Lua exception scheme. A solution is sketched in
the next section.

=== A complete solution ===

== Outline ==

{{{
catch (struct lua_longjump * p_lj) {
  // force local status to non 0
}
catch (...) {
  // do soft unwinding
  throw
}
}}}

NOTE: `p_lj` should be pointing to the current `lj`.

TODO: assert that.

This should guarantee that we don't eat alien exceptions, unless
someone twisted decides to throw `(struct lua_longjump)*` all over the
place from her library.

== Soft unwinding ==

To devise our algorithm, we need to identify the vital resources that
are acquired and released by `pcall`.

'''Outline of the pcall stack'''
        * top level: caller (e.g. application code)
        * lapi.c pcall (the API entry point)
        * (lua_lock: currently empty)
        * /!\ ldo.c D_pcall has some serious '''cleanup code''' !
        * ldo.c luaD_rawrunprotected (where the exception is caught)
        * bottom level: actual C function call (where the exception is
thrown on Lua)

The top and bottom levels wrap when pcalls are nested.

'''The D_pcall cleanup'''
{{{
if (status != 0) {  /* an error occurred? */
  StkId oldtop = restorestack(L, old_top);
  luaF_close(L, oldtop);  /* close eventual pending closures */
  luaD_seterrorobj(L, status, oldtop);
  L->nCcalls = oldnCcalls;
  L->ci = restoreci(L, old_ci);
  L->base = L->ci->base;
  L->savedpc = L->ci->savedpc;
  L->allowhook = old_allowhooks;
  restore_stack_limit(L);
}
L->errfunc = old_errfunc;
}}}

It restores the state previously acquired:

{{{
// ptrdiff_t old_top, ptrdiff_t ef
unsigned short oldnCcalls = L->nCcalls;
ptrdiff_t old_ci = saveci(L, L->ci);
lu_byte old_allowhooks = L->allowhook;
ptrdiff_t old_errfunc = L->errfunc;
L->errfunc = ef;
}}}

Now, we could use the ''Resource Acquisition Is Initialization''
pattern to perform cleanup code automatically at object destruction.
It is the natural way to integrate with exceptions (Stroustrup
14.4.1).

== Coding roadmap ==

        * start with current algorithm, refactoring as RAII; test
        * use empty soft unwinding, relying on RAII; test

=== Final notes ===

Stroustrup 14.9 is a fundamental.



-- Thomas