On Thu, Nov 2, 2017 at 8:41 AM, Marc Balmer <marc@msys.ch> wrote:
> Is it possible to create a luaL_Buffer over several calls to Lua functions? E.g. like in the following pseudo code:
[...]
> Or must all buffer operations be finished when I return from C code?
I never used that before, so it was interesting for me to understand how that works.
I have learnt that (with Lua 5.3) luaL_Buffer has an initial buffer space as a field. When, however, the user wants to add more data than the initial buffer can hold, a userdatum pointing to a new buffer space is allocated and pushed on top of the stack. Subsequent buffer operations assume that the userdatum is at the top of the stack, without much checking. The userdatum is removed from the top of the stack only by luaL_pushresult(). As far as I can tell, this means that once C-code starts using a buffer, it must call luaL_pushresilt() before returning to Lua, otherwise the userdatum that may or may not have been placed on top of the stack becomes its return value. If the buffer manipulation straddles C/Lua boundary, then things can get even stranger.
In fact, it is unsafe to do any manipulations of the Lua stack between luaL_buffinit() and luaL_pushresilt(). The following code (with Lua 5.3) prints 1 and finishes normally:
lua_State *L = luaL_newstate();
luaL_Buffer lb;
lua_pushinteger(L, 1);
luaL_buffinit(L, &lb);
luaL_prepbuffsize(&lb, 5);
luaL_addsize(&lb, 5);
printf("%d\n", (int)lua_tointeger(L, -1));
lua_pushinteger(L, 1);
luaL_prepbuffsize(&lb, 5);
luaL_addsize(&lb, 5);
The following code (with Lua 5.3) prints 0 and then crashes in the last line:
lua_State *L = luaL_newstate();
luaL_Buffer lb;
lua_pushinteger(L, 1);
luaL_buffinit(L, &lb);
luaL_prepbuffsize(&lb, sizeof lb.initb + 1);
luaL_addsize(&lb, sizeof lb.initb + 1);
printf("%d\n", (int)lua_tointeger(L, -1));
lua_pushinteger(L, 1);
luaL_prepbuffsize(&lb, sizeof lb.initb + 1);
The manual does indicate that what the code above does is invalid, but it uses fairly mild language that can be easily misunderstood as something benign:
During its normal operation, a string buffer uses a variable number of stack slots. So, while using a buffer, you cannot assume that you know where the top of the stack is. You can use the stack between successive calls to buffer operations as long as that use is balanced; that is, when you call a buffer operation, the stack is at the same level it was immediately after the previous buffer operation.
(end)
I am unsure why such a dangerous facility is present, but I think that its dangers should be stressed by stronger language. The problem is that the crash happens only when large strings are stuffed into the buffer, which is a perfect ingredient for "it works when I test it, but it crashes randomly in production, and no one knows why".
I think it can also be made less dangerous by using a reference to the userdatum rather than the top of the stack, but frankly, I think this is something that should only be used by Lua itself and its public use deprecated.
