Re: RE: Pluto: heavy-duty persistence for Lua

[benjamin sunshine-hill <bsunshin@...>]

Thanks for the input!

> 2. If closures hold you back on support for Heavy Userdata, you could
> instead rely on tables.  This would mean that a userdata's serialization
> metamethod should return a table rather than a closure.  This table can then
> be persisted in the userdata's place.  A likely method for the
> deserialization would be for this "stand in" table to include a
> deserialization metamethod.  This is/was my current design, but I like your
> closure approach better.

As I write this, I'm in the middle of hammering down Lua closures. It's a lot easier than I thought it would be. Alas and hooray, my girlfriend is returning from vacation this afternoon, so it may be a couple of days before I polish it off. ;)

> 3. Tables should be checked for a serialization metamethod just like
> userdata.  The reason for this is that table's are Lua side "objects" or
> "type extensions" and may therefore need extended handling compared to a
> "primitive" type.  Examples of such "needs" would be for proper handling of
> "cached" values after deserialization to account for changes in the
> environment, simply tossing out those cached values to save space in the
> stream, reloading/linking of libraries, etc.

Huh, I hadn't thought of this. You're absolutely right, of course...anything with a metatable should be able to use it for customizable persistence.

My current idea was to have three possiblities for the metatable for userdata: mt.__persist==true, in which case it would be literally serialized like a string, mt.persist==function, returning the closure, or no value, indicating that the userdata may not be persisted. I'll probably do the same thing for tables, except that the default will allow literal persisting, requiring an explicit "false" value to prevent persistence. How's that sound?

> 4. Closures can also represent an "extended type" in Lua, but I have
> absolutely no clue on how they should be given "special consideration" in
> the way that heavy userdata and tables should.  Others who frequent the list
> can probably make useful comments on this issue.  However, I'd recommend
> keeping this in mind, but not stalling anything because of it.  Why?  Just
> because I personally wouldn't know what to do with it and therefore I
> believe most people wouldn't either. (and yes I am the center of my own
> universe :-)

My gut feeling is that as long as the upvalues are persisted correctly, including those that potentially invoke metatable persistence methods, it'll "just work". Anyone who can think of a counterexample, please pipe up.

> 5. In my own serialization of heavy userdata I found it necessary to add my
> own "special type" called a blob (aren't I so original).  My blob's were
> very much like Lua strings in that they had a length and could store any
> binary data, but they weren't allocated through Lua, they did NOT have Lua's
> string value comparison semantics, and they did NOT require performing a
> copy operation to place them on the Lua stack.  Basically this amounts to an
> arbitrary block of data in the stream, that a userdata can use to store its
> internal data in whatever format it wishes (in my case this included a lot
> of large pictures saved in the format used by a particular camera SDK).  A
> userdata could include one of these in its "stand-in" table to be
> serialized.

Hmm... so what, then, was the difference between a blob and a heavy userdata?

Ben