I'll answer for ftools/fsyntax. ;)
First of all, since this is kind of a middle-man between 'raw' token filters, and metalua, let's have a short presentation.
- age: a few weeks
- maturity: allows multiple filters to co-exist (ftools), parses Lua syntax (fsyntax), seemingly bugless.
- complexity: between raw filters and metalua (1095 lines with comments)
- what are their overhead, when used, when not used?
Loading time of life.lua seems to go up max 30 times (from plain Lua). That's with syntax parsing, but no modifications. Whether you'd care depends on whether your user would notice.
Used when syntax sugar is needed (fsugar has all my favourite patches).
Usage: lua -lftools -lfsugar -lfsyntax -lfsugar2
That is, there's clearly two levels of filtering tools here. Ftools provides your sugar files an agstraction with :read(), :rewind(), :replace(), :flush() and such. It knows none of Lua syntax, and can be used (only) for local syntax sugar issues. Most of them happen to be in this category, though.
Fsyntax adds onto that (and onto the runtime cost, thus kept separate), reproducing the _whole_ Lua 5.1 syntax in a combo of functions, tables, and string constants. It's basically a tube to pour your code through, and will subtube it to appropriate places. This capability brings it rather close to Metalua, but not quite.
With little expansion, fsyntax could also generate the bytecodes (which Metalua does), or some other bytecodes for that matter? Java VM? Microsoft? Anyways, I won't go that far but you may..
Make me headache? :P No, honestly ftools is maybe designed, fsyntax "just happened". One of the how far can you go -issues, that sometimes take place. And you're unable to stop. One more... Truly, I have no idea where this will lead. There may be honor, glory, success, or World War III. Maybe.
Ruin a perfectly Good language? :P ehem...
hmm... Fabien had a lot in this section, but I would be more optimistic. Many things he says can't be done, I'd say yes, they can.
Fundamental difference between ML and fsyntax approaches is, ML does a structured representation of the syntax, whereas fsyntax is merely the 'tube'. Extending syntax is made by injecting changed syntax definitions, including 'checkpoint' functions that mark certain places in the stream (kind of like gsub captures). No tree form is ever done of the token stream, even part of it.
Needs to be measured. I'd be very eager to see the results.
Fabien said it well, noting no source that does not compile with vanilla Lua shall be called lua. I'll also be thinking of explicitly needing to state the used filters within the file (currently not needed) to shovel some uncertainties out. The risks are as big as having cpp in standard C. So are the benefits.
( Roberto said in LuaOhSix that for the first time, he wouldn't know, which are the important features for the next Lua version (5.2). To me, it's becoming more and more obvious token filters, in one form or the other, would be the Big Next Thing. There's a certain amount of playing involved to see, what exactly rocks, and what not, and then it's up to the authors to make sense out of it all. But my hunch is this will be in for real. )
Same here! :) Thanks a lot for asking.
Grellier, Thierry kirjoitti 29.11.2006 kello 14.50: