[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: Is it possible to add utf-8 lua source file support in lua 5.2?
- From: Luiz Henrique de Figueiredo <lhf@...>
- Date: Tue, 28 Sep 2010 06:38:53 -0300
> The utf-8 bom, by definition of unicode, is actually a "space" character.
> Shall we just treat utf-8 bom like a normal space character, instead of
> strip it off? Is that easier to handle in the lexer?
In Lua 5.2 you don't even have to patch the lexer: just edit lctype.c
and say that 0xFF and 0xFE are whitespace. This of course is not the
perfect solution, because BOM is a 2-byte entity, not a 1-byte one...