Talk by Roberto: 'Novelties of Lua 5.2'

[steve donovan <steve.j.donovan@...>]

Hi all,

I found this excellent talk by Roberto by accident when considering
Lua 5.2 for a project:

http://www.inf.puc-rio.br/~roberto/talks/novelties-5.2.pdf

Well worth the read, but what's interesting is the last section: what
was left out.

The first is actually a proposed removal, automatic coercions.

The second is some kind of macro facility.  The last slide is the
interesting one, which discusses 'macros in the large'; how to
harmonize the use of macros with the development of larger
applications.  So it's about scoping and loading, modules,
precompiling and appropriate error messages.  Classic (stupid) C
macros would clearly be a regression, with no scoping or module
system.

These seem to me to be classic questions of policy. The Lua spirit is
to define the _minimal_ mechanisms needed to support required
policies. Once we have those, the rest can be done in the usual spirit
of discussion and Darwinian selection ;)

For scanning and lexical analysis, we already have an excellent tool
in LPeg, so I could move LuaMacro from a hard dependency on the
token-filter patch to a soft dependency on LPeg thanks to Peter
Odding's Lua lexer.  MoonScript defines its full grammar as well with
LPeg.

On another evolutionary branch, we have Fabien's Metalua (which
remains a bytecode compiler) which was based on Kein-Man Hong's
Yueliang (a Lua 5 front-end written in Lua)  And we also have David
Given's Objective Lua.

The minimum extra feature needed would probably be our old friend the
#line directive. Together with goto, this makes it a lot easier for
tools generating Lua to compile to _self-running_ Lua code.
Currently, with LuaMacro and MoonScript, you need to run the converted
code using a script that can sort out of the line number mappings - if
you want reasonable error messages. Both tools can of course generate
standalone Lua code but the line number information is lost.
(Curiously enough, I found that LuaMacro was more useful as a C
preprocessor (e.g. the source for winapi) due to those magic #line
directives.)

I think Mike Pall is open to some kind of #line, but he points out
that it could be  costly to save _arbitrary_ file/line information.
Joshua Jensen has been thinking about solving this problem more
efficiently.

As for the discussion points, I've tried to define some policies with
LuaMacro.  _Lexically scoped_ macros can be defined, and there is a
require_ macro (note the underscore) which is evaluated immediately by
the preprocessor and so can define macros for the current file.
(There's also an include_ which works like #include).  Files with a
given extension (e.g. .luam) can be automatically converted on the fly
(which is also how MoonScript does it).

(Curiously enough, I'm personally agnostic about macros. My enthusiasm
for custom syntax is tempered by my suspicion of arbitrarily defined
syntax ;)  But there are cases where it can really work well, like
with custom DSL generation for specialized tasks.)

As you can see, a veritable who's who of the Lua universe has been
involved in this (and I haven't even mentioned lhf for his original
token-filter patch)

steve d.