Re: UTF-8 identifiers [was: Re: Lua t-shirts]

[Jerome Vuarand <jerome.vuarand@...>]

2009/10/5 Ico <lua@zevv.nl>:
>
> * On 2009-10-05 David Given <dg@cowlark.com> wrote  :
>
>> I'm still pushing for my proposal to allow all characters with codes
>> in the range 128-255 to be considered valid in names.  This would
>> allow identifiers contain UTF-8 sequences.
>
> Interesting idea. What's the catch, if there is one ?

Lua lexer uses isalpha C function, therefore depending on the locale
some of the characters in 128-255 are already valid. You could change
it to allow all characters in 128-255 in addition to 'alpha' class and
underscore, but that would be a hack imho.

On the other hand, as far as I can tell making the lexer unicode-aware
(so that you can use iwalpha) would require a major overhaul,
especially since Lua strings are not strings of characters but string
of bytes (and are often used to hold non-textual binary data).