Re: Modifying the parser to store a function definition

[Peter Cawley <lua@...>]

You may be interested in the persistence library which I wrote for a
game project:
Documentation: http://code.google.com/p/corsix-th/wiki/Persistance
Source: http://code.google.com/p/corsix-th/source/browse/trunk/CorsixTH/Src/persist_lua.cpp

In particular, the handling of functions is probably similar to what
you want. The workflow looks something like:
1) Load Lua files with persist.dofile rather than dofile.
2) persist.dofile loads the file into memory, and passes it into a
normal unmodified luaL_loadbuffer. If the load fails, then the error
is returned to the caller, otherwise:
3) Scan for functions marked with "--[[persistable:name]]", note down
the line number, pull out the source code from the "(" of the argument
list to the matching "end", stick on a "function " prefix and an
appropriate number of newlines to keep line numbers the same, then
save the following:
 a) filename, line number => persistable name
 b) persistable name => filename
 c) persistable name => the piece of source code
4) Return the loaded chunk to the caller

Persisting a function looks like:
1) Get the filename and line number via the debug library.
2) Do sanity checks to ensure that we don't have a C function, and
that the function was seen by persist.dofile
3) Write the persistable name, gotten via previously saved
information, and the upvalue names, gotten via the debug library.
4) Write the values of upvalues and of the function environment

Depersisting a function looks like:
1) Read the persistable name and the upvalue names
2) Construct the Lua chunk:
local upvalue_name_1, upvalue_name_2 --[[ etc. ]] ; return
function(...) upvalue_name_1, upvalue_name_2 --[[ etc. ]] = ... end,
--[[ previously saved piece of source code]]
3) Load this chunk, giving it the same name as was previously saved,
and then call this chunk.
4) Read the values of the upvalues, then call the first of the
returned functions, passing these values.
5) Return the second of the returned functions.

I choose to write the persistable name rather than the source code to
allow source code to be patched without having to go into each
savegame and also patch code there. If you don't consider this worth
the code of having to put a persistable decorator on functions, then
the source code could be persisted instead. If you go down that route,
then you'd probably want to delay scanning source files for functions
until you actually need to persist said functions, at which point
you'd load the source file again, go to the appropriate line, find
"function", then find the next "(" and pull out everything up to the
matching "end" (you know the source file is valid, so try loading up
to the first "end", then up to the second "end", etc., until it
loads). You probably wouldn't want to save the filename or preserve
line numbers, as these things may not match up any more after you load
the savegame.

On Thu, Jul 8, 2010 at 5:50 PM, Nicolas <nicolas@net-core.org> wrote:
> Hi,
>
> In my game I make heavy use of having functions inside data structures
> which I need to serialize.
> Currently I am simply string.dump'ing the functions and it works fine.
> However this is not portable between architectures, luajit2 does not
> yet support it, and so on.
>
> So I thought I could modify the parser to save the string that defined
> a function inside the function structure itself so I could later use it.
> I do not care for closures and upvalues in such cases so I only need the
> code (and string.dump does not handle them anyway).
>
> I quickly found I do not sufficiently know the parser to do that yet, so
> has anybody done such a thing?
>
> The tricky part is that it needs to be remembered as just a function, so
> that I.E:
> function foo(a) print("foo", a) end
>
> is remembered as just:
> function(a) print("foo", a) end
>
> So just using ls->lineumber is invalid, it needs the exact position of the
> parenthesis.
> I tried ls->current for the position but they seem very much off :/
>
> I am open to ideas :)
>
> Thanks
>