Re: Is it possible to add utf-8 lua source file support in lua 5.2?

[Mike Pall <mikelu-1009@...>]

Robert Raschke wrote:
> 2010/9/27 J.Jørgen von Bargen <jjvb.primus@gmx.de>
> > I doubt, this will not work on pipes. (characters are lost?)
>
> Indeed, if you want to strip a UTF-8 BOM from a pipe stream you need to do
> something else. Essentially, you'd need to add in a read ahead buffer.
> That's a bit more work than the 10 extra lines for ditching a file BOM.

There's already such a buffer, but it's inside the lexer. That's
why I've moved the '#'-line detection from lauxlib.c into the core
of LuaJIT. I've also added UTF-8 BOM stripping, because trying to
educate Windows users _not_ to use notepad.exe is an absolutely
hopeless endeavor. The LuaJIT lexer has been modified to accept
non-ASCII UTF-8 characters as identifiers, too.

[
Yes, I prefer pragmatic solutions ('It helps my user base') over
dogmatic approaches (aka 'Someone is wrong on the Internet!' or
'Someone might have a ternary, one's complement computer without
IEEE-754-compliant floating-point hardware somewhere in his shed').
See also: http://xkcd.com/386/
]

> I haven't looked in detail, but do pipe opens even go via luaL_loadfile() ?

Well, yes, if you use a named pipe. Whether that's a good idea or
not is a different issue.

--Mike