RE: Userdata/Table Unification

["Virgil Smith" <Virgil@...>]

'Userdata/Table Unification' definitely gets my vote as well.

My primary reasons for supporting this measure are...

1. It greatly simplifies handling per instance data for userdata.
2. It improves Lua's orthogonality (which means it makes Lua conceptually
simpler and more efficient, and these improvements would probably apply to
Lua's VM code as well).

My primary concern is...

1. Sandboxing.  The fact that Lua code cannot modify the metatable of
Userdata is an important security benefit.  Consider for instance the effect
of a script changing the gc metamethod of a userdata.  On the other hand,
Lua extends a method for Lua side code to mark a metatable as protected and
so this is duplicated effort.  Also, the most important aspect of script
security <i.e. sandboxing> is not protected by the "special status" of
userdata.  That aspect is the protection of variables that reference
userdata values so that malicious/buggy scripts can't "hijack" the variable.


-----------
Comments about Mike's statements...


- Tables do not have a __gc() metamethod.
I am in complete agreement that tables should have __gc metamethods, but
their lack was a deliberate decision on the part of the Lua authors.  When I
commented on this lack previously <reasonably early in the 5.0 lifecycle>
the only responses I received were that they felt __gc metamethods were not
needed by Lua tables.  However, I've seen this come up on the list several
times as "the need to use a userdata proxy" in order to obtain a __gc
metamethod.




- Current userdata objects are allocated by the Lua core and cannot be
  resized nor moved nor shared. Quite a few libraries end up storing
  a single void* in a userdata object. This is wasteful.
  Leaving the allocation management to the C library seems to be the better
  solution in the long term IMHO.

The Lua authors stated reasons for this handling is to remove the need for
__gc metamethods in "most cases".  However, I for one have never made a
userdata that didn't need a __gc metamethod.  In fact I would expect that
most userdatas that allocated no resources other than memory should simply
be tables so that their values are directly accessible to Lua code.  Anyone
have counter examples?

In the past I would have greatly appreciated a mechanism for reporting the
size of a userdata to Lua so that the gc triggered properly, but I expect
this will be irrelevant with the incremental gc.  BTW: Allocating userdata
through Lua is only a partial solution for this because of allocations done
internally by userdata's.




-----------
As always, my sincere thanks to the Lua authors and those who have helped
along the way.  This is a marvelous language.  In no way should my
suggestions for small tweaks be taken as disapproval of the work that has
been done in the past.  Thanks especially for offering your labors up to
world for use and abuse!!






-----Original Message-----
From: lua-bounces@bazar2.conectiva.com.br
[mailto:lua-bounces@bazar2.conectiva.com.br]On Behalf Of Mike Pall
Sent: Monday, July 26, 2004 6:51 AM
To: Lua list
Subject: Userdata/Table Unification


Hi,

Edgar Toernig wrote (in the thread 'Lua 5 and tags'):
> Jamie Webb wrote:
> > I'd like to add that adding an array part to userdatas would also
> > provide fast typing:
>
> Hmm... that gets me back to the old idea of merging userdata and
> tables: give every table an additional void* field and remove the
> userdata type completely *g*

Maybe it's time to start a 'Userdata/Table Unification' thread ...

The same topic pops up every so often. Summing up all advantages and
disadvantages in one thread may be helpful. And it may be a first step
in convincing the authors of Lua to add this to the core.

Just a few thoughts to start the discussion:

- Adding a void* to the table structure incurs no additional memory
  overhead on most systems. I.e. the userdata pointer is free!
  The current table structure is 32 bytes long (on 32 bit systems) and
  anything between 29 and 36 bytes leads to the same effective memory
  allocation due to malloc overhead and alignment constraints. YMMV.

- A plain userdata object has 16 bytes (on 32 bit systems) plus the
  userdata itself plus any malloc and alignment overhead.
  The savings provided by a trivial userdata object over a unified
  userdata/table object are negligible. If you really need to save space
  there is still lightuserdata (I think this type is valuable and should
  not be affected by the unification).
  More complex libraries actually need less memory because they no longer
  need table wrappers, auxiliary tables or per-object metatables.

- I cannot see any noticeable speed difference between the current approach
  and the unified approach (method access, userdata pointer access, type
  checking ...).

- Current userdata objects are allocated by the Lua core and cannot be
  resized nor moved nor shared. Quite a few libraries end up storing
  a single void* in a userdata object. This is wasteful.
  Leaving the allocation management to the C library seems to be the better
  solution in the long term IMHO.

- Many libraries need to store additional Lua objects along with userdata
  values and end up abusing the metatable or weak tables. This complicates
  type checking and/or garbage collection.

- Tables do not have a __gc() metamethod. This prevents wrapping userdata
  objects into a table because __gc() metamethods often need to have
  a look at the full object context and not just at individual userdata
  objects. Unifying userdata and tables would provide such a __gc()
  metamethod for tables. This is advantageous even for pure Lua code.

- Current userdata objects cannot be subclassed easily. Unless you have
  the cooperation of the C library you need to wrap them up in a table and
  store any subclass fields there. But this approach is troublesome
  due to the lack of a __gc() metamethod for tables. See above.
  Subclassing gets trivial with a unified userdata/table object.

- Every unified table object has a void*. This may be useful for many more
  scopes than just plain library objects (e.g. module namespace tables).

- I have not taken a closer look at the issues that the unification rises
  for the Lua garbage collector. I may be wrong, but I think it gets simpler
  because the tricky handling for userdata objects can be dropped.

- Reducing diversity usually simplifies the implementation and this leads
  to fewer bugs. Some people believe this is the only measure that counts.

Bye,
     Mike