Calling Complex C Functions

[Tim Hill <drtimhill@...>]

(Apologies for the longish post…)

One issue we face when using Lua is coding what we term a “complex” C function (probably not the right name, but whatever). By “complex” I mean specifically one that is stateful and/or calls a significant number of helper functions to manipulate that state. Now, the typical design pattern for such a function is to wrap the state into a C structure (on the stack or heap), and then pass that state around as necessary (much as Lua uses lua_State in its own API). Pretty much a no-brainer I think.

But things get nasty when that state is on the heap, or has indirect pointers to items on the heap or file handles etc that need to be cleaned up when the C function returns to Lua. The problem, of course, is that pretty much all the Lua API reserves the right to throw a Lua error (for example, when calling a metamethod or when out of memory). This abruptly terminates the C function execution without giving it a chance to perform any state cleanup, resulting in leaked memory, open file handles and/or dangling state etc.

it seems to me that at present there are only a few work-arounds for this:

[A] Design the C function to not use state, or not call any Lua APIs while it holds resources/state. This results in very strange contorted code even when it is possible.

[B] Compile Lua for C++ so that it uses exceptions and not longjmp, and then use try() clauses to inject “finally” handling to clean up resources. This of course is only viable when C++ is available, and assumes that Lua uses exceptions in a manner that allows that to work (I’ve not studied this in detail).

[C] Allocate the stateful structure as a Lua full userdata, and then use __gc() to perform cleanup when Lua collects the abandoned structure (essentially a dispose() model).

[D] Have a dummy “outer” C function that then does a lua_pcall() into the (inner) real C function, so that Lua errors are caught and the state can be cleaned up before the outer C function either returns or re-throws the Lua error.

In our case, both [A] and [B] aren’t really viable for the reasons noted, which leaves [C] or [D]. The problem with [C] is the non-deterministic garbage collect (the dispose pattern problem, basically) and the overhead of userdata/metatable/__gc compared to a simple C structure on the stack. Nasty things can also happen if (for example) file handles are sitting around waiting to be closed at a GC. So basically I’m stuck with using [D], unless anyone else has a better idea.

But [D] has it’s problems also. The first is efficiency; doing the lua_pcall() has the overhead of creating yet another Lua stack frame etc etc. it’s also pretty clumsy, and involves chunks of extraneous plumbing code, which is quite outside the usual clean Lua API model. To me, it just doesn’t feel “right”.

So, the more I think about it, the more I think this is really a hole in the Lua API. My feeling is that quite a lot of C functions would be “complex” (using my definition), and would benefit from some help from Lua to manage cleanup of state should Lua decide to raise an error. What these functions need is a (kind of) “finally” mechanism; they don’t want to intercept errors, they just want to be told when an error has occurred while still in context. Something like this:

-----
typedef void (*lua_OnErrorFunction)(lua_State *L, void *p);
lua_onerror(lua_State *L, lua_OnErrorFunction f, void *p);

Registers an error handler function for the current C function call. If Lua throws an error within the current C function call, calls f passing p as the sole argument. This call occurs before Lua demolishes the C stack when it performs a longjmp. There can only be one error handling function for a given C function call; calling the function multiple times in a given C function call replaces any existing handler. Note that the error handler is registered only for current invocation of the C function, and thus must be called each time the C function in called.
-----

This is basically similar in concept to the way Lua handles continuation context for coroutines. I’ve not studied the Lua source extensively, but my casual reading is that the overhead of the proposed facility would be confined to checking for registered handler function(s) when an error was raised, which presumably is infrequent enough in normal code execution to not cause any significant performance hit. There would of course be some additional memory overhead to maintain the registered handler state against each C function stack frame.

I can (of course) see some issues with this approach:
— Since C functions can nest (even recursively) each called C function needs its own error handler, so when an error is raised there would potentially be several handlers to call (presumably from the most recent to the oldest).
— I suspect the handler would be very limited in the allowable Lua APIs that could be safely called (perhaps none), since raising an error inside a handler should not be allowed. This is probably the nastiest issue with this model.
— This is not really a “finally” clause at all, since it’s not called on a clean C function exit, only on an error (this is not necessarily a defect to the design).
— There is no error context information passed to the handler function (again, I don’t see this as necessarily an error).

Something like this API would dramatically clean up some of the awkward code we have in our project, and I suspect would help many others too. Of course, it adds complexity to Lua, but without it coding anything other than trivial C functions seems rather more complicated that it really should be. Unless I’m missing something obvious?

(btw, I’m aware that i can code something resembling the above via a helper C function that does a lua_pcall() to simulate the handler, but this has all the issues outlined in [D] above.)

—Tim