Re: Performance (was Re: Selenophobia)

[Daurnimator <quae@...>]

On 27 March 2017 at 11:59, Tim Hill <drtimhill@gmail.com> wrote:
> #2 of course can be mitigated with a __gc finalizer but now things get complex, as you must: allocate memory via full userdata, setup a metatable for that userdata, push a C function onto the stack as the finalizer, set it as __gc, and set the metatable on the userdata. That’s a fair chunk of code. And of course you are subject to the usual caveats about when the finalizer will actually execute.

Turns out this is pretty easy in practice, see e.g.
https://github.com/daurnimator/lua-psl/blob/98c47f0a62d68144a5c07cc9887e45e90e989246/psl/psl.c#L245
You can abstract this too, see e.g.
https://github.com/wahern/luaossl/blob/ae16dd4dd147404fc73e34ab3263d50d93a57f0b/src/openssl.c#L474


> What is missing here of course if some kind of “finally” equivalent, since any C function that worries about leaks is really going to break down into three phases:
>
> 1. Setup and allocate resources as necessary.
> 2. Perform some operation(s) that might throw an error.
> 3. FINALLY cleanup the resources from step 1 and either return success or re-throw the error.
>
> Now, you CAN do this in Lua, but it’s pretty messy since step 2 needs to be wrapped in a lua_pcall() so that the errors can be trapped for step 3. But you can only use lua_pcall() to call Lua C functions, so you have to allocate resources in step 1, then create the necessary call stack to lua_pcall() the C function at step 2 (passing through arguments from the original call, plus the allocated light/full userdata). Yet again, this all adds up to a fair bit of code.
>
> When we first encountered these issues, I had a developer look across a range of Lua rocks to see what others had done. The results were disturbing. In at least 50% of the rocks, we found code that would leak resources since the code simply assumed (or hoped) Lua would NOT throw an error. (I suspect much of this code was simply written without a clear understanding that Lua could indeed throw these errors.)
>
> Of course we DID work around these issues, using some of the techniques described above. But, as one of my developers said, "we seem to be fighting hard here against the Lua API and I’m not sure why”.
>
> So how could things be better? To my mind, the ideal would be a new Lua C API that allowed a lua_pcall() to a C function directly:
>
> typedef int (ProtCFunc)(lua_State *L, void *p);
> int lua_pcallc(lua_State *L, ProtCFunc* func, void *p);
>
> Makes the call "func(L, p)” in a protected environment, catching any errors thrown. If func() returns normally, the return value is the return value of func(), otherwise it is the same as for lua_pcall(). Note that no new Lua stack frame is setup; the called function will see the same Lua stack as the original C function (and can push return values if it desires).
>
> Now I can see all sorts of issues here (for example, can you make additional lua_pcallc() calls from within a protected C function?), but this is a much more efficient way of handling resource management for C functions than either of the work-arounds above (assuming of course that setting up such a protected environment doesn’t add huge overhead and is practical).

You mean like the (now deprecated) lua_cpcall?
https://www.lua.org/manual/5.1/manual.html#lua_cpcall
This can be replicated in lua 5.3 with:

lua_pushlightuserdata(L, p);
lua_pcall(L, 1, 0, 0);