Re: Ideas about implementing a string type?

[Rici Lake <lua@...>]

David Kastrup wrote:
> Hans Hagen <pragma@wxs.nl> writes:
> > indeed, and returning values instead of a table in the associated
> > > functions may also speed up the process; but simplicity and staying
> > > close to lua's way of doing things is also worth a price; concerning
> > > the tokenizer callback (which is what this is about) ... using
> > > numbers instead of strings (them being hashed or simple strings)
> > > makes sense anyway since in tex they are numbers; there are related
> > > functions that map them to symbolic names (strings);
>
> Sure: I already said that it was one possibility to pass basically
> opaque data (and ids that have to be resolved into strings by separate
> functions are opaque) into Lua.  It just makes things less natural.

In the case of tokens, it's easy to use precreated Lua strings. You
simply fill a reference (Lua) table with the strings, indexed by
the token number, and attach the table to the function as an
upvalue. Then you can pass Lua the (Lua) string by doing:

   lua_rawgeti(L, lua_upvalueindex(2), token_number);
   /* 2 is just an example. Usually I use 1 for the metatable */

which involves very little work, and is about twice as fast
as lua_pushliteral(). Perhaps it's a bit more than twice as
fast; certainly, it's fast enough to be noticeable, but not
fast enough to be worthwhile except in cases where it is
done a lot. In the case of a lexer callback, it's certainly
worth doing, and the above is precisely how I do it.