RE: Pluto updated

["Virgil Smith" <Virgil@...>]

Go Ben go!  Sorry (again) that I haven't had time to work on this.

One difference in implementation structure I noticed in Pluto as opposed to
what I've done in the past is the 2 stage behavior.  The "reference table"
need not be passed explicitly in the stream.  It can instead be spread
throughout the stream.  This eliminates the need for having 2 separate
stages for "__persist" specialization.

In my implementation this merely meant "counting" objects as they were
serialized/deserialized.  As each object was serialized I performed a check
to see if it had already been serialized and if it was then its "count" was
retrieved in the process and could then be used as a reference.  Upon
deserialization the list of objects was formed in reverse and any references
that came out of the stream necessarily had to refer to already deserialized
objects.

Another advantage of this approach is that references need not conflict with
light-userdata (or any other type) as they can be their own "type" in the
stream which thus signaled that they are a reference and not a "target
object".  These values then of course are translated to LUA_NUMBER for use
in the tables of serialized/deserialized objects.

A distinct disadvantage to this approach is that it does not integrate as
directly with Pluto's table of "permanents" (sorry I've forgotten the term
you used, but I am referring to your table of items that are assumed to be
present in the deserialization environment).

However, this still isn't too much of a departure from the Pluto high level
stream components.  Currently (or rather last I knew) Pluto serializes the
"references table" and then the reference that indicates the top level
serialized object.  The switch would just be that first the "permanents"
table is serialized, and then the top level object (rather than just a
reference to it).

Of course this would involve a change in the code architecture that is not
at all insignificant.  Also, I feel like I'm missing some impact on the
operation of the "permanents" table.



-----Original Message-----
From: lua-bounces@bazar2.conectiva.com.br
[mailto:lua-bounces@bazar2.conectiva.com.br]On Behalf Of benjamin
sunshine-hill
Sent: Tuesday, June 15, 2004 11:00 AM
To: Lua list
Subject: Re: Pluto updated


Okeydoke, why not. I'll make it a conditional compilation option.

Now, one caveat that needs to be worked out. __persist metamethods are
invoked during the first stage of persistence (so that all needed objects
referenced by the closure can be visited) but nothing is written out until
the second stage (since not all references are assigned until then). So in
order for this to work, Pluto will have to invoke a special stage-2 persist
metamethod during stage 2.

Here's my idea: For userdata that need this special behavior, set the
__persist mt member to "true". Then, set some other member
"__persistlowlevel" to the function to execute during stage 2. The writer
and ud will be pushed on the stack as lightuserdata.

Unpersisting is a bit more tricky, since we have to know, somehow, which
userdata to invoke low-level unpersist methods for, and what methods to
invoke for them. This needs to happen in stage 1 of unpersisting (since
that's when we read everything). Normal unpersist methods are executed after
stage 2. Ideas?

Ben

----- Original Message -----
From: Grisha <grisha@eagle.ru>
Date: Tuesday, June 15, 2004 5:10 am
Subject: Re: Pluto updated

> Hi, Ben!
>
> We have such a system up and running, and it's not that complex at all, I
> can send it to you if you wish. You are right, Pluto doesn't have to know
> anything about how C++ objects are serialized, same as C-side must know
> nothing about how Pluto does it's job. What I'm asking for is just to
> recieve those context parameters passed to Pluto (reader/writer & (void
> *ud)) back in persist/unpersist methods. This will not complicate things
> much for either you, as Pluto developer, or any other programmer working
> with Pluto (they will just ignore extra parameters).
>
> -- modified text from Pluto\readme
> Here's an example of special persistence for an object:
>
> vec = CreateMyObject()
> setmetatable(vec, { __persist = function(oldtbl, writer, ud)
> local mt = getmetatable(oldtbl)
> local state = DoCSideSaving(oldtbl, writer, ud)
>
> return function(newtbl, reader, ud)
>  setmetatable(newtbl, mt)
>  DoCSideLoading(newtbl, reader, ud, state)
> end
> end })
>
> Note that original example will work too, since it will just ignore extra
> parameters. We really need this, we can live without function or thread
> persistance, but without proper uderdata persistance we can't.
>
> Thanks in advance,
> Grisha
>
> ----- Original Message -----
> From: "benjamin sunshine-hill" <bsunshin@usc.edu>
> To: "Lua list" <lua@bazar2.conectiva.com.br>
> Sent: Saturday, June 12, 2004 1:17 AM
> Subject: Re: Pluto updated
>
>
> > Hmmm..... it's a tricky issue that you raise. After all, Pluto's
designed
> to serialize a bunch of interreferencing Lua objects, rather
> interreferencing C++ objects. A system capable of serializing C++ objects
in
> a parallel manner would be complex indeed. Still, I really think that the
> current situation will work. What would need to be done would be for the
> __persist functions to invoke the C++ persistence framework in such a way
> that temporary Lua objects--userdata, most likely--were created to
> encapsulate the object state. If objects are shared between different
> userdata (using some custom C-side reference counting mechanism, for
> instance), some registry-related method could be used to coordinate
> serialization between different C object entry points. This is, of course,
a
> rather complex method... but it's really not much more complexity than
> absolutely necessary to serialize a C++ framework. Pluto would still
> automatically handle fixup before running the unpersistence method
> > s, of course.
> >
> > Ultimately, I don't think that the main question is whether the
__persist
> metamethod writes through a string/userdata or through a writer object;
> that's just semantics, with some performance implications. The real
question
> is how much control __persist can exert over the serialization stage, and
> how much control it has over internal Pluto placeholder references. I'd
> really prefer to keep those as hidden implementation details; they've
> already gone through a couple of revisions, and there are quite a few
> caveats that they deal with in order to keep Lua happy.
> >
> > Ben
> >
> > P.S. I'm currently working on the string-keys-for-permanents idea you
> proposed. It's proving to be more difficult conceptually than I'd
expected,
> to maintain flexibility and provide as much user-control as possible, but
I
> should have it ironed out in a week or so (travel will be slowing me
down).
> >
>
>