RE: Extending Lua in C, was RE: External modules

["Brownsword, Andrew" <brownsword@...>]

While I agree that Lua is simple and easy to modify, doing so means that you need to reapply or reimplement the modifications everytime the Lua team revs their code.  With other code modules this gets pretty expensive and is to be avoided, but perhaps Lua is simple enough that it doesn't matter.

Thanks for the code snippets below, looks like there are some interesting ideas there.

Cheers,
  Andrew


-----Original Message-----
From: Nick Trout [mailto:ntrout@rockstarvancouver.com]
Sent: Friday, November 15, 2002 7:07 PM
To: Multiple recipients of list
Subject: Extending Lua in C, was RE: External modules



> Brownsword, Andrew
> Sent: November 15, 2002 1:38 PM
> Currently we have to change the Lua distribution 
> to extend the language, but this could be solved by an 
> dynamic extension mechanism.  Support for specific platform 
> DLL implementations can then be provided by code that isn't 
> distributed as part of the core Lua library.

I believe Reuben Thomas suggested extending Lua using "metaprogramming"
and componentising Lua quite some time ago. In addition a few have
suggested componentising the garbage collection area of Lua (although
this was mentioned as being impractical). I don't think anything has
come of the suggestions because 1) it is widely accepted that Lua is
small and uncomplicated enough to modify for your own purposes and 2) it
challenges one of the design goals, that Lua be very simple. However, I
believe your proposal only extends to adding basic data types to Lua at
code level, not altering or expanding the syntax of the language. eg.
adding vector types, RGBA colour, array accessors etc.

If we look at the binary addition operator, its optimised for number
types, and any other types are dealt with using metatables (from Lua
5.0alpha lvm.c). So the problem is its easy to invent usertypes and
operator overload in script, but you have to use the same mechanism in
C, which is not so efficient.


StkId luaV_execute (lua_State *L) {
     ...
     case OP_ADD: {
        StkId rb = RB(i);
        StkId rc = RKC(i);
        if (ttisnumber(rb) && ttisnumber(rc)) {
          setnvalue(ra, nvalue(rb) + nvalue(rc));
        }
        else
          Arith(L, ra, rb, rc, TM_ADD);
        break;
      }



static void Arith (lua_State *L, StkId ra, StkId rb, StkId rc, TMS op) {
  TObject tempb, tempc;
  const TObject *b, *c;
  if ((b = luaV_tonumber(rb, &tempb)) != NULL &&
      (c = luaV_tonumber(rc, &tempc)) != NULL) {
    switch (op) {
      case TM_ADD: setnvalue(ra, nvalue(b) + nvalue(c)); break;
      case TM_SUB: setnvalue(ra, nvalue(b) - nvalue(c)); break;
      case TM_MUL: setnvalue(ra, nvalue(b) * nvalue(c)); break;
      case TM_DIV: setnvalue(ra, nvalue(b) / nvalue(c)); break;
      case TM_POW: {
        const TObject *f = luaH_getstr(hvalue(registry(L)),
                                       G(L)->tmname[TM_POW]);
        ptrdiff_t res = savestack(L, ra);
        if (!ttisfunction(f))
          luaG_runerror(L, "`pow' (for `^' operator) is not a
function");
        callTMres(L, f, b, c);
        ra = restorestack(L, res);  /* previous call may change stack */
        setobj(ra, L->top);
        break;
      }
      default: lua_assert(0); break;
    }
  }
  else if (!call_binTM(L, rb, rc, ra, op))
    luaG_aritherror(L, rb, rc);
}


static int call_binTM (lua_State *L, const TObject *p1, const TObject
*p2,
                       TObject *res, TMS event) {
  ptrdiff_t result = savestack(L, res);
  const TObject *tm = luaT_gettmbyobj(L, p1, event);  /* try first
operand */
  if (ttisnil(tm))
    tm = luaT_gettmbyobj(L, p2, event);  /* try second operand */
  if (!ttisfunction(tm)) return 0;
  callTMres(L, tm, p1, p2);
  res = restorestack(L, result);  /* previous call may change stack */
  setobj(res, L->top);
  return 1;
}


If there was a mechanism for registering user type numbers, eg. with a
certain range, which seems perfectly possible:

typedef union {
  void *p;
  union TString *ts;
  union Udata *u;
  union Closure *cl;
  struct Table *h;
  lua_Number n;
  int b;
} Value;


typedef struct lua_TObject {
  int tt;
  Value value;
} TObject;

#define ttisnil(o)	(ttype(o) == LUA_TNIL)
#define ttisnumber(o)	(ttype(o) == LUA_TNUMBER)
#define ttisstring(o)	(ttype(o) == LUA_TSTRING)
#define ttistable(o)	(ttype(o) == LUA_TTABLE)
#define ttisfunction(o)	(ttype(o) == LUA_TFUNCTION)
#define ttisboolean(o)	(ttype(o) == LUA_TBOOLEAN)
#define ttisuserdata(o)	(ttype(o) == LUA_TUSERDATA)
#define ttislightuserdata(o)	(ttype(o) == LUA_TLIGHTUSERDATA)

Perhaps the VM could be altered slightly. Lets assume user types are
>1000 :-

StkId luaV_execute (lua_State *L) {
     ...
     case OP_ADD: {
        StkId rb = RB(i);
        StkId rc = RKC(i);
        if (ttisnumber(rb) && ttisnumber(rc)) {
          setnvalue(ra, nvalue(rb) + nvalue(rc));
        }

        else if (<type number bigger than 1000>)
          /* can just pull user type handler callback from an array */
          <Call user registered callback for this 
               type>](L, ra, rb, rc, TM_ADD);

        else
          Arith(L, ra, rb, rc, TM_ADD);
        break;
      }

The following would be in the user extension lib:

void <User registered type handler> (lua_State *L, StkId ra, StkId rb,
StkId rc, TMS op) {
    <b and c are the operands, a the result>
    <check a,b and c are vectors (or whatever)>
    switch (op) {
      /* the following might use C++ vector operator overloads */
      case TM_ADD: setnvalue(ra, nvalue(b) + nvalue(c)); break; 
      case TM_SUB: setnvalue(ra, nvalue(b) - nvalue(c)); break;
      case TM_MUL: setnvalue(ra, nvalue(b) * nvalue(c)); break;
      case TM_DIV: setnvalue(ra, nvalue(b) / nvalue(c)); break;
      default: lua_assert(0); break;
    }
  }
}

Sorry I don't time to write a patch and try this, its just a suggestion.

I don't think Lua will ever be a metaprogramming language that you can
alter syntactically on the fly. However I believe extending Lua as
mentioned is within the bounds of the design goals. Indeed making Lua
faster is one of the primary design goals.

Nick