Re: lua_tolstring and wide character strings

[Jerome Vuarand <jerome.vuarand@...>]

2009/8/14 Shmuel Zeigerman <shmuz@013net.net>:
> Recently I was manipulating wide character strings in a Lua library. Windows
> API assumes these strings to end with L'\0' which is represented as 2
> successive zero bytes. The problem is, lua_tolstring guarantees only 1 zero
> byte at the end (according to the manual):
>
>> lua_tolstring returns a fully aligned pointer to a string inside the Lua
>> state. This string always has a zero ('\0') after its last character (as in
>> C), but can contain other zeros in its body.
>
> If I solve this by appending L'\0' to strings (which is expensive
> operation), then I have problems to lua_concat such strings.
>
> So I've a wish for Lua 5.2:
>    "Make lua_tolstring guarantee 2 zero bytes at the string end".
> It seems that no noticeable overhead would be incurred, and no compatibility
> issues would arise.
>
> Please correct me if I'm wrong, or if better options exist.

AFAIK there is little reason to use UTF-16 in Lua rather than UTF-8.
With UTF-8 most string library functions can be used to some extent.
There are nice functions in the win32 API to convert from UTF-8 to
UTF-16 and back : MultiByteToWideChar and WideCharToMultiByte.