Re: Feature request: userdata slice

[William Ahern <william@...>]

On Wed, Aug 19, 2015 at 07:21:42AM +0000, 云风 Cloud Wu wrote:
> >   AN ADDRESS IS NOT AN INTEGER!
> >
> >   I remember this crap going from 16 bit to 32 bit systems [1] and frankly,
> > I'd rather not have to live through that again going to 64 bit [3][4].
> >
> 
> An address in not an integer, but a handle can be.
> For example, in posix , file handle is an integer , and file is an object
> ,too.
> 
> We don't need create more than 2^32 userdata in one lua vm, so we can
> create an array in lua global state ( G(L) ),  and use an integer index
> to indicate the userdata object's address.
> 
> It would be like this:
> 
> union Value {
>   GCObject *gc;    /* collectable objects */
>   void *p;         /* light userdata */
>   int b;           /* booleans */
>   lua_CFunction f; /* light C functions */
>   lua_Integer i;   /* integer numbers */
>   lua_Number n;    /* float numbers */
>   struct {
>       int handle;
>       int slice ;
>    } u;   /* full userdata with an integer */
> };

This seems needlessly complicated.

Firstly, I don't agree that 2^32 handles is enough. Lua tables might be
limited to 2^32-1 entries, but you can have many tables. Rather, if anything
2^32 slices is too much, at least in the scenarios you've presented. (OTOH,
slices of C userdata arrays might prove quite popular, and being limited to
2^32 would _definitely_ be problematic.)

Secondly, the pointer indirection and handle management seems unnecessary. A
slice is an annotation on the underlying value, similar in some respects to
it's type. Have you tried storing the slice data with the value type tag
(TValue), instead? There appear to be over 20 unused bits.

For true array slices you'll need to double or maybe even triple the size of
a TValue, anyhow, so you can store a vector. Short of that, repurposing the
type tag bits would seem more than sufficient. And it would work just as
well on 32-bit platforms as 64-bit platforms--IIUC your proposed solution
limits you to 16-bit tuples on 32-bit machines.

I actually started this e-mail to criticize the whole idea, but then I
started to see the problems and possibilities when playing around with
metatables on lightuserdata out of curiosity. Then it all clicked. :)

Still, as a practical matter I think (as do most others, probably including
yourself) that bindings which expose the low-level details (such as
hierarchy) of C data structures are a poor idea in general. For my money the
better remedy for all the garbage generated by object.x.y.z indexing is to
improve the semantics of the application's Lua and C libraries so there's
fewer discrete loads and stores of C member values in the Lua code,
especially in critical areas.

Even absent the garbage objects, all those indexing operations aren't
helping performance, and this solution doesn't remedy that at all. Using
(abusing?) Lua as a convenient interface to poke and prod C data structures
is a poor use of both Lua and C. And I think that's why, fundamentally, this
notion of slices is problematic--it's most useful for all the wrong reasons.
(Except, perhaps, for array slices, but your proposal doesn't satisfactorily
solve that.)

If there's simply no way to refactor the code there's always LuaJIT FFI.