Re: Is it possible to add utf-8 lua source file support in lua 5.2?

[Robert Raschke <rtrlists@...>]

On Tue, Sep 28, 2010 at 10:38 AM, Luiz Henrique de Figueiredo <lhf@tecgraf.puc-rio.br> wrote:

> The utf-8 bom, by definition of unicode, is actually a "space" character.
> Shall we just treat utf-8 bom like a normal space character, instead of
> strip it off? Is that easier to handle in the lexer?

In Lua 5.2 you don't even have to patch the lexer: just edit lctype.c
and say that 0xFF and 0xFE are whitespace. This of course is not the
perfect solution, because BOM is a 2-byte entity, not a 1-byte one...

Unfortunately, a UTF-8 BOM is 0xEF 0xBB 0xBF.

Robby