Re: luaL_loadfile doesn't like the UTF-8 BOM

[Jerome Vuarand <jerome.vuarand@...>]

2012/7/9 Simon Orde <sorde@gotadsl.co.uk>:
> Script authors can also write Lua modules, however, and by default these are
> loaded by luaL_loadfile, which doesn't like the UTF8-BOM and throws an
> error.   This isn't actually that serious a problem, because one simple
> solution would be for me to simply specify that modules must always be
> encoded in ANSI.  However, if possible, I would prefer to allow modules to
> be encoded in either UTF-8 or ANSI too (the rule about string encoding
> matching script encoding would not apply to modules).  Can anyone suggest a
> way that I can do this, while still retaining the UTF-8 BOM?  I had a look
> at the package.loaders section in the manual, but this seems to only provide
> a way to have a module-specific loader, whereas my requirement applies to
> all modules.

"package.loaders" doesn't contain module-specific loaders, that's the
role of "package.preload". "package.loaders" is an array of
"searchers", which are functions that take a module name and return
the module initialization function (which is called a "loader"). These
"searchers" and "loaders" are used by the require function.

You can write a "searcher" that look at the BOM, skip it, and compile
the module (but does not run it) with the proper load function. I put
an example of a "searcher" written in Lua on the wiki a while ago:

http://lua-users.org/wiki/LuaModulesLoader

What you have to do is replace the basic "return
assert(loadstring(assert(file:read("*a")), filename))" with something
specific to your situation (BOM skipping). For examples of "searchers"
written in C look a the Lua source code (IIRC Lua itself has 4
predefined "searchers").

Note that all this applies to Lua 5.1, but I believe it wasn't changed
much in 5.2.