RE: Pluto updated

["Virgil Smith" <Virgil@...>]

> Not exactly: note the 'nice' word in my statement :). This approach will
work, but will require unnecessary copy of undefined amount of data several
times, which IMHO looks ugly. It would be much more convinient for
programmers to serialize userdata in it's native form, without converting it
into Lua form.

The problem with this suggestion is that it gets ugly itself with respect to
object length encoding/termination in the resulting stream.  Lots of objects
don't know their serialized size until after serialization is complete.  Any
such object must either accept a copy operation in the process OR requires
use of a terminator(a.k.a. sentinel) value/sequence, which in turn must be
guaranteed not to occur in the stream or an escaping mechanism must be
added. <Actually there is another option, but it violates the stream concept
and so does not fit with the reader/writer function abstractualization, i.e.
back up in the "stream" to record the length.>

It seems inappropriate to me for Pluto to enforce a terminator/escaping
mechanism.  So it would have to rely on C code associated with the userdata
to enforce its own mechanism, which requires that the userdata must add some
form of identification into the stream BEFORE it serializes, and then if
that identification cannot be matched to something in the Lua universe
during the unpersist operation then the whole remainder of the stream will
be lost.

IMHO this complexity is not advisable for heavily reused code.  I think it
would be better to keep the mainline code as simple and intelligible as
possible and then let users customize the code to handle specialized needs.


NOTE: There is 1 additional case to be mentioned (and I think its the case
that Grisha was discussing).  The naively simple serialization method for C
structures, etc. is to just binary copy them (or parts of them) into the
stream.  In this case the current lua_pushuserdata interface is pointlessly
inefficient because it will require a pointless raw copy operation.  This is
not a fault of Pluto so much as a falt in Lua5 (see the "Lua and C memory
management" thread).  A "light" userdata is useless as a buffer pointer
because it does not include size information.  A "full" userdata requires C
side objects to be tied to Lua immediately upon creation, or force a copy
operation.  What would be handy here is something like a "middleweight"
userdata that contains a pointer and a size.  A __persist metamethod could
create these rather than create full userdata and copy the data into it.

Of course it would be very nice if Lua5.1 "fixed" userdata in this way, then
Pluto would not have to add its own special case "type".  On the other hand,
usage consistent with a contiguous buffer of the specified length could not
be enforced by Lua, and it would even be advantageous for some GC purposes
if this was NOT the norm (unless userdata also get a useful per-instance
data mechanism in which to keep a full list of buffer/length pairs to cover
all dynamically allocated members).