lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

On Tue, Sep 28, 2010 at 10:38 AM, Luiz Henrique de Figueiredo <> wrote:
> The utf-8 bom, by definition of unicode, is actually a "space" character.
> Shall we just treat utf-8 bom like a normal space character, instead of
> strip it off? Is that easier to handle in the lexer?

In Lua 5.2 you don't even have to patch the lexer: just edit lctype.c
and say that 0xFF and 0xFE are whitespace. This of course is not the
perfect solution, because BOM is a 2-byte entity, not a 1-byte one...

Unfortunately, a UTF-8 BOM is 0xEF 0xBB 0xBF.