Re: Using Lua's string interning for faster compares?

[Matthew Wild <mwild1@...>]

On 20 January 2012 02:03, Jay Carlson <nop@nop.com> wrote:
> [Obligatory suggestion for Lua core, if people insist on playing with C strings. Like pushlstring: lua_findlstring(L,s,l), which will return false if s would have had to be allocated. If s would have to be allocated it cannot possibly be a key in a raw table. This optimization could be applied to a lua_rawgetfield() without anybody noticing.]
>
> [Additional string functions in Lua could support this as well: I have a long string from the network; give me my command to look up, or give me nil if it's pointless to try to look up in any table. Well, rawget from any table. __index can find anything. I find rawget annoying, and I don't know if I want to encourage it.]
>
> On Jan 17, 2012, at 7:04 AM, Peter Cawley wrote:
>
>> In one of my current projects, I have a similar need to frequently
>> switch over a Lua string. I opted for a solution of hash-based
>> dispatch combined with code generation, which I don't think I've seen
>> proposed on lua-l recently.
>
> I have two reactions:
>
> First, Lua tables seem to work well in most circumstances. I would consider restructuring my code after writing about the second C string switch statement. I come to Lua to escape from C and C strings, and I intend to fully enjoy my vacation.
>
> My second reaction is that, uh, actually I've been in that switch statement situation with Lua tables before. My ancient FLTK binding used tolua++, which produces a lua-side metatable for every class. You can just eat that on desktop platforms, but I was working on an 8M Linux PDA; every app built with the FLTK binding had ~300k of *unsharable* allocated memory by the time it made it to main. A lot of it might not be touched. With swap, that kind of thing is an annoyance; without, private memory gradually squeezes the rest of the system to death. And the payload, the actual working Lua FLTK app, was often like ~100k of private memory. All of that was with custom gc settings to minimize the amount of sbrk.
>
> IMO that's the level of measurement and desperation appropriate for bailing out on tables. I didn't, but I did have a plan.
>
> I had an advantage: most of my tables were already being generated from tolua++ source. I didn't need to scan my own source code to find strings, and there already was a code generation step. What I thought I needed was rommable hash tables, some const structure which could live in the .text/.rodata section and be sharable (and a candidate for page-out) across all the apps loading the luafltk shared object.
>
> My plan was to create my own user type with an __index which could look up methods by name in a (perhaps perfect) const hash table generated by tolua++. This C hash would not be visible to users; a regular writable Lua table would be hit first. At that point I could play around with memoizing the C-side hash results, weak tables etc.

Similar idea to eLua's rotables:
http://www.eluaproject.net/doc/v0.8/en_arch_ltr.html

Regards,
Matthew

PS. Congratulations to the eLua team on the new website, first time
I've seen it... looks good, comprehensive, and I still managed to find
that page in just a few clicks :)