lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

On Fri, Jan 16, 2015 at 4:53 AM, Rob Kendrick <> wrote:
> On Fri, Jan 16, 2015 at 12:11:41PM +0000, Aapo Talvensaari wrote:
>> Is it by design that utf.len count the BOM to length?
>> Say utf8.len("\xEF\xBB\xBFa") will return 2 instead of 1?
> Given UTF8 has only one valid "byte order", it makes no sense to ever
> include a byte order marker in a UTF8 document.

Sure it does -- the UTF-8 BOM is used (and aggressively promoted by
Microsoft) as a magic number to identify the contents of the file as
UTF-8 text. The XML spec even explicitly supports this (although many
XML parsers do not).

/s/ Adam