Experimental finalize/guard patch

["alex.mania@..." <alex.mania@...>]

Ok, as far as I can tell the first email got lost in transit. Either that or
emails with attachments take longer than 7(!) hours to go through. So at the risk
of a double post, here it is again:

Hey all,

As unwind_protect, try finallys, RAII all seems to be the topic of the week I
thought I'd try my hand at implementing such a thing in to the parser/vm. Thanks
to Hu Qiwei for his recent try finally patch, this patch borrows some ideas from it.

I am yet to migrate over to Lua 5.1.3, and have a few other patches on my copy of
Lua, so I had to copy the changed code over to a 5.1.3 before making the diff.
Hopefully I haven't left any code out (I should have got it all - I delimit all
patches inside #ifdefs etc), and hopefully it just.works. Would appreciate
feedback on whether this is the case.

I chose to implement unwind_protect's through two keywords, "finalize" and
"guard". A finalize block will always get run after the code it protects, a guard
only gets run in case of error. I have no idea what the de facto standard
keywords are, and would love suggestions.

Where I use the patch mostly is grabbing critical sections and in
beginupdate/endupdate pairings, eg like the following:

| function appendud(tab, ud)
|   tab:beginupdate() finalize tab:endupdate() end
|   ud:lock() finalize ud:unlock() end
|   for i = 1,#ud do
|     tab[#tab+1] = ud[i]
|   end
| end

(The equivalent Lua code would be monstrous, but here it is with Hu's try finally
patch:)

| function appendud(tab, ud)
|   tab:beginupdate()
|   try
|     ud:lock()
|     try
|       for i = 1,#ud do
|         tab[#tab+1] = ud[i]
|       end
|     finally
|       ud:unlock()
|     end
|   finally
|     tab:endupdate()
|   end
| end

Although I don't aim to debate whether upside down code prevents errors or not,
personally I think the former is far less likely to introduce bugs as a missed
pairing is easy to spot, as is one not protected by a finalize.

The example from http://www.ddj.com/cpp/184403758 would look like this:

| function user:addfriend(newfriend)
|   self.friends:pushback(newfriend) guard self.friends:popback() end
|   database:addfriend(self:getname(), newfriend:getname())
| end

(which is equivalent to)

| function user:addfriend(newfriend)
|   self.friends:pushback(newfriend)
|   try
|     database:addfriend(self:getname(), newfriend:getname())
|   catch err do
|     self.friends:popback()
|     error(err)
|   end
| end

Anyway, let me know what you find.

For those interested, here are some notes on the implementation:

(1) The code is not rearranged to execution order. I don't know if it would speed
it up much, or at all, to do so. I do know it would not be fun though - the code
needs to be moved, the line numbers need to be moved, the variable declarations
need to be moved, breaks out of the code need to be corrected... good luck.

(2) Finalizers/Guards are run in reverse order, eg:
| function test()
|   print "a" finalize print "A" end
|   print "b" finalize print "B" end
|   print "c" finalize print "C" end
| end
would print "abcCBA"

(3) Break and return are completely supported, eg:
| function test()
|   for i = 1,2 do
|     print "a" finalize print(i) end
|     break
|   end
| end
would print "a1", and
| function test(...)
|   finalize print {...} end
|   return "a", "b", ...
| end
| print(test(1, 2, 3))
would print "{1,2,3}", "a", "b"
This is complicated, it means there are 4 ways through a finalize block (normal,
break, return, and error). The implementation provides this by reserving a
position on the lua stack for finalize flow, and loading a constant when
non-normal flow is required. This is fairly flexible, to enable continues/break n
patches would not be too hard.
The advantage of this approach is it allows the vm to nest calls without using
additional C stack (as is standard in Lua). The disadvantage is that a runtime
check has to be made to ensure that the stack hasn't been modified.

(4) Temporary returns are held on the stack between varargs and the local stack
(similar to Hu's implementation). This requires two memmove calls. A possible
optimization would be to reserve space for say, 2 results, and any more then that
perform the move. (not implemented)

(5) Proper lexical scoping is provided, with any local variables declared before
the guard/finalize being free to be modified/accessed. There's one caveat: when
using a guard/finalize the "until" in a repeat loop can no longer access local
variables declared after the guard/finalize.

(6) The standard luaD_pcall function is used. This should allow it to work in
C++, delphi, C, anything. Longjmps/Setjmps are not required. Two stack positions
are used per guard/finalize, and a bit of C stack. The C stack is unavoidable, as
the current implementation of Lua only allows protected calls if the code to be
protected is wrapped inside a function call. (nCcalls is checked for C stack
overflows). The biggest disadvantage of this is that coroutines cannot yield from
inside a protected region, without using Coco (a feature of luajit). Note, I have
not tested coco, but believe it'll work.

(7) Only four opcodes are added, some dual use, all preceded by OP_TRY. These are:
OP_TRYENTER (luaD_pcalls the vm, starting on the first instruction after the guard)
OP_TRYCLOSE (closes a protected vm, resuming flow from the TRYENTER)
OP_TRYRESUME (either rethrows, jumps to the default dest, or jumps to break/etc)
OP_TRYRETURN (if ARG_C actually returns, else moves returns and closes vm)

I think that's about it. If there's any questions, suggestions or bugs please shout.

Oh, and note that whether a finalize or a guard is used, the error is always
rethrown. It wasn't my goal to reimplement Lua's error handling (pcall works fine
for me), just provide a way to guarantee atomicity in functions and ensure
releasing of resources.

- Alex

Attachment: guards.zip
Description: Binary data