Re: Feature request: userdata slice

[William Ahern <william@...>]

On Wed, Aug 19, 2015 at 02:48:37PM +0000, 云风 Cloud Wu wrote:
> >
> > Even absent the garbage objects, all those indexing operations aren't
> > helping performance, and this solution doesn't remedy that at all. Using
> > (abusing?) Lua as a convenient interface to poke and prod C data structures
> > is a poor use of both Lua and C. And I think that's why, fundamentally,
> > this
> > notion of slices is problematic--it's most useful for all the wrong
> > reasons.
> > (Except, perhaps, for array slices, but your proposal doesn't
> > satisfactorily
> > solve that.)
> >
> 
> So slice is not a good name, let's change a name.
> 
> I think one integer can use to present the array slices. We don't need use
> it for begin and end position of slice, just record the index of slices.
> The slices of array is also store in userdata. I don't think you will
> create 2^32 slice of one userdata.

What if I have a data structure like:

	struct foo {
		struct {
			int a;
			int b;
		} list[];
	};

where list could be a billion items long? That's not at all uncommon in data
mining. Even multi-gigabyte XML and JSON files are increasingly common.

Your initial proposal treated slices as offsets, but by limiting slices to
32-bit integers on 64-bit machines they don't actually work that way. Unless
the width of a slice identifier is at least as wide as the addressable
region, we can't reference arbitrary fields of arbitrary userdata values.
That's an intolerable inconsistency in semantics.

If we exclude the notion of slices as offsets or indices, what we're left
with is a notion of tagging or qualifying userdata values. Maybe the tag
means the value is immutable (without needing to create a proxy object), or
maybe it means it's a view on the member .x.y (when we can encode that in
the limited space).

In that case, there's no need for this system of handles. 32 bits is as
useful (or useless) as 8 bits for limited tagging ability. Just use the free
bits in the type member of TValue.