Re: Lua's symbol table

[Edgar Toernig <froese@...>]

Thatcher Ulrich wrote:
> 
> On Sat, 4 Aug 2001, Edgar Toernig wrote:
> 
> > Ehem... another note: this property of string pointers is afaik not
> > documented.
> >
> > > Essentially what I want is something like:
> > >
> > > int     get_symbol_id( const char* name );
> > > const char*     get_symbol_name( int id );
> > > void    lua_getglobal_using_id( lua_State* L, int id );
> > > etc.
> >
> > This can be even done without knowing about the above mentioned
> > property of string pointers.  Use lua_ref/getref to map strings
> > to integers.
> 
> OK, I think I get this pretty well.  Although if I want to keep that char*
> valid, it doesn't appear that Lua makes any guarantees, other than that
> lua_getref() on the locked reference will still push a valid string.  It
> seems as though the garbage collector is (and should be) still free to
> move the actual data around.

As I said, this property is not documented but in all versions of Lua
I know (3.x, 4.x) the pointer stays the same until there's no longer
a reference to it.  Lua's garbage collector does not move data around.

IMHO if you embed Lua into some application you can make use of this
property.  But if you want to take future memory management changes
into account...

> >  If you call get_symbol_id more then once for a given
> > name you have to manage a lookup table though to not create
> > multiple references for the same name.
> 
> This part I'm not clear on... if I do this:
> 
>         lua_pushstring( L, "test" );
>         int     ref1 = lua_ref( L, 1 );
>         lua_getref( L, ref1 );
>         int     ref2 = lua_ref( L, 1 );
>         lua_pushstring( L, "test" );
>         int     ref3 = lua_ref( L, 1 );
> 
> Are ref1, ref2 and ref3 guaranteed to be equal?  Seems like they should
> be, but it's not spelled out in the manual.  That's what I need for the
> symbol-table idea to be useful.

They are _not_ equal.  Each call to lua_ref returns a new number.  Only
after calling lua_unref this number will be reused.  That's why I said
you should use a lookup table.  Something like this:

    static int table_ref;
    ...
    // during startup create the lookup table
    lua_newtable(L, 0);
    table_ref = lua_ref(L, 1);
    ...
    int
    getid(lua_State *L, const char *str)
    {
        int id;

        lua_pushstring(L, str);
        lua_getref(L, table_ref);

        lua_pushvalue(L, -2);
        lua_rawget(L, -2);
        if (lua_isnil(L, -1)) // new string
        {
            lua_pushvalue(L, -3);  // create new ref
            id = lua_ref(L, 1);

            lua_pushvalue(L, -3);  // and store string/ref in table
            lua_pushnumber(L, id);
            lua_rawset(L, -5);
         }
         else  // ref already in table
            id = lua_tonumber(L, -1);

         lua_pop(L, 3);
         return id;
    }

Although this is the documented way, it will only work on Lua 4.x.
3.x has a different API.  And who knows what will happen in 5.x?
So IMHO it's OK to use the mentioned pointer property in cases where
it makes sense.  I.e:

    const char *field1_name;
    const char *field2_name;
    const char *field3_name;
    ...
    static const char *
    get_lua_ptr(lua_State *L, const char *str)
    {
        const char *p;
        lua_pushstring(L, str);
        p = lua_tostring(L, -1);
        (void)lua_ref(L, 1);
        return p;
    }
    // during init
    field1_name = get_lua_ptr(L, "field1");
    field2_name = get_lua_ptr(L, "field2");
    field3_name = get_lua_ptr(L, "field3");
    ...
    // and then later i.e.:
    int
    foo_gettable_callback(lua_State *L)
    {
        const char *name = lua_tostring(L, 2);

        if (name == field1_name)
             ...
        else if (name == field2_name)
             ...
        else if (name == field3_name)
             ...
        else
             luaL_verror(L, "bad field `%s'", name);
        return 1;
    }

Sure, it's not the most elegant way, but afaik the fastest and that's
what this whole thread is about...

Ciao, ET.


PS: Lua 4.1 does not recycle reference numbers.  That may give problems
with long running applications that make heavy use of references.
After 256^sizeof(int)/2 lua_ref calls bad things will happen...