Re: Pluto generating source code?

[Peter Cawley <lua@...>]

On Mon, Oct 31, 2011 at 1:30 PM, Stefan Reich
<stefan.reich.maker.of.eye@googlemail.com> wrote:
> Outputting source code would be quite elegant and certainly more
> readable than bytecode.

Indeed, though doing so within the 5.1 API is rather tricky. My
bespoke approach to persistence, along with reasons for not using
Pluto, is available at
http://code.google.com/p/corsix-th/wiki/Persistance. As with any
persistence library, the hard bits are:
* Light userdata. One approach is to treat them as bit strings, and
upon de-serialisation deal with differing endianness and widths. The
other common approach is a permanent objects table.
* C functions. The only really viable approach is a permanent objects table.
* Lua functions. One approach is bytecode, but I am acutely aware of
how dangerous this is, so one alternative is to use source code. If
doing this, you need some method to go from a function object back to
source code. The debug library can tell you the "filename" and line
number of a function object, so if you were watching as everything was
loaded then you can find the source code of that "filename", and find
the appropriate line on that file. If you subsequently impose a
restriction of no more than one function per line, then you can
unambiguously pull out the source code. My persistence library takes
roughly this approach, but serialises a unique token placed alongside
the function source code rather than the function source code itself.
This is done to allow code and serialised data to evolve differently,
but wouldn't be too hard to change it. In any case, shared (strong)
upvalues are a little tricky. You can detect them using the 5.1 API
(set two upvalues to different values, then test if the two upvalues
contain the same value; more efficient variations of this exist), or
using the 5.2 API (just call lua_upvalueid). Glueing them back
together upon de-serialisation is another problem.
* Full userdata. The general approach is to have some metamethods
responsible for [de]serialisation. In the case of Pluto, said
metamethod is meant to return a representation of the userdata as some
other Lua object which doesn't involve userdata. I went for a
different speed / robustness tradeoff, and let the metamethods read or
write whatever stream of bytes they want.
* Coroutines. The 5.1 debug library is probably good enough to perform
the required introspection. As for reconstruction, Pluto delves into
the Lua internals, and I fear that this is the only viable approach.