Re: What's up with token filters (Re: New operators?)

[Philipp Janda <siffiejoe@...>]

Hi!

Asko Kauppi schrieb:
> 
> My first reaction to Andy's "token filters only in C" approach was
> -yack, they're so easy to do in Lua-.  But in a way being able to
> compile the _interpreter_ patched, without touching the lexer / library
> side would indeed grant most of the benefits. And it would clearly
> state, which one people are using (think: lua51, lua51-continue,
> lua51-incdec). But so would: lua51, lua51 -lcontinue -lincdec.

I think token filters should not be activated on the command line but
instead separately for each chunk. This way you could use modules with
different syntax extensions. So how about separating token filter
loading and token filter activating? For loading one could reuse the
existing require-mechanism. The hard part is the activation of token
filters. One way would be to add the token filters to require, dofile
etc. as parameters. This would probably result in lots of wrapper files
and IMHO the necessary token filters should be mentioned in the chunk
source anyway. So I would consider a small syntax extension for
activating token filters for the given chunk (similar to the shebang
line? maybe a special comment?). In this case a C api way of specifying
default token filters (need to be preloaded) could be used to tune Lua
for specialized tasks.


Philipp