Re: Serializing Lua functions (was Re: status and perceptions of luaproc)

[Tim Hill <drtimhill@...>]

> On Aug 7, 2015, at 3:38 PM, Sean Conner <sean@conman.org> wrote:
> 
> It was thus said that the Great Rena once stated:
> 
>  Actually, you can serialize Lua functions with upvalues.  I've done it
> (just not shown the code).  In fact, I was able to serialize quite a bit,
> including tables with circular references.  The only things I couldn't
> handle were coroutines, user data and Lua functions written in C [1].
> 
>  As it stands, the function serialization is only good across the same
> architecture (everything else I could serialize was CPU independent) and I
> started working on fixing that by diving into the "dump format" used by Lua.
> 
>  -spc (I kind of lost interest when I realized I didn't have a need for
> 	this)
> 
> [1]	At least not directly.  I could "serialize" the function io.write(),
> 	but I did so by reference, assuming the receiver side could resolve
> 	such a reference.  I could even handle known userdata types like
> 	io.stdout by again, using references. [2]
> 
> [2]	Strings basically.  I just passed the name along.
> 
> 

Yes we did the same thing recently for a project here. Ended up with a nice home-grown algorithm that can do any Lua value except (as you noted) threads, userdata and C functions. Single pass that handles arbitrary table graphs, upvalues (with handling for upvalues shared between closures) etc. We also did the work to make the bytecode mostly portable, handling integer width and endian differences, but not differences in floating point format.

The only thing we could not handle cleanly was _ENV, since there is no explciit information on this provided by the compiler. My workaround was to *define* an environment table as one in which T[“_G’] == T, and use this to detect upvalues that were env tables (note that you CANNOT assume the first upvalue is the _ENV table). This assumption is dubious at best, but in practice it works in 99% of the scenarios we could come up with.

—Tim