[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: Is it possible to add utf-8 lua source file support in lua 5.2?
- From: Robert Raschke <rtrlists@...>
- Date: Tue, 28 Sep 2010 10:49:08 +0100
On Tue, Sep 28, 2010 at 10:38 AM, Luiz Henrique de Figueiredo <email@example.com>
> The utf-8 bom, by definition of unicode, is actually a "space" character.In Lua 5.2 you don't even have to patch the lexer: just edit lctype.c
> Shall we just treat utf-8 bom like a normal space character, instead of
> strip it off? Is that easier to handle in the lexer?
and say that 0xFF and 0xFE are whitespace. This of course is not the
perfect solution, because BOM is a 2-byte entity, not a 1-byte one...
Unfortunately, a UTF-8 BOM is 0xEF 0xBB 0xBF.