Re: static strings [was Re: Lua Compile/one time Load?]

[Tom Sutcliffe <tomsci@...>]

> On 20 Oct 2023, at 14:49, Roberto Ierusalimschy <roberto@inf.puc-rio.br> wrote:
> 
>> I would like to see the third subtype of lua string : external string
>> . The content of the string is a C pointer rather than bytes array.
>> So that we can construct the long strings in O(1), it's useful when we
>> want to use lua string library to manage the memory in C side.
>> Or if we have many lua vm in one process, it can reduce the cost of
>> message passing.
> 
> For a more generic "C pointer string subtype" in Lua, it probably would
> need some system to release that external memory, e.g. by associating
> the string with a callback to be called when the string is collected.
> 
> A fixed string could use a NULL callback. Such system could also allow
> the buffer created by 'luaL_Buffer' to be used directly as the string
> content in Lua, solving another issue mentioned previously (memory
> duplication when instantiating the final string in a luaL_Buffer).

So long as that could still avoid the allocation for the TString (or equivalent), ie the whole operation only needed 1 allocation in total?

> 
> However, if we introduce a callback, it probably would also need a user
> data to be passed to the callback. So, each external string would need
> three pointers of overhead. That could be significant for "short" long
> strings (such as in the range 40~100 bytes).

I'm not sure I can think of a case where I'd really need or want a callback, let alone one stored on a per-string basis, that does seem perhaps a little excessive. Unless the data structure was such that if the callback was NULL those extra pointers' worth of memory wasn't allocated (like how lua_newuserdatauv behaves when nuvalue=0)?

Just having the ability to specify external data that the caller implicitly guarantees will remain valid for the lifetime of the lua_State would be enough for me. Since the original poster's requirement was regarding module code, which also (almost always) has a lifetime of the state as well, seems like that would be sufficient?

Cheers,

Tom