Re: What's up with token filters (Re: New operators?)

["Andy Stark" <AStark@...>]

Lua list <lua@bazar2.conectiva.com.br> writes:

>How about getting token filters into the game, and then _reducing_ even
>the currently existing sugar (: notation, function name() etc.) into a
>joint default token filter (let's say -llua51), and out of the C lexer.

>This will make _any_ sugar outside of the language itself.  In a way, it
>puts them all into equal position (dmz).

I think there is a solution that will offer most of the advantages of
token filters without most of the dangers: add a token filtering system to
the Lua parser but only make it available via a straightforward C API (ie,
don't allow filters to be written in Lua at all). Arrange this API so that
it can't be used from libraries loaded at runtime, but only from code
built into the interpreter. Then, implement the suggestion above: remove
existing syntactic sugar from the core language and recreate it using this
new token filter system.

If this was done, then the sugar included with the core would be easy to
remove from the language when necessary, but including it wouldn't incur a
significant time penalty for people who are content to use it. It would
allow experimentation with new language features, but wouldn't allow the
creation of third party libraries with non-standard syntax. Embedders
could create special syntax for their own purposes as appropriate.

Perhaps this could be done just for the *next* version of Lua with the
option of adding pure Lua token filters in future versions. By then, we
would have some gentle experience to inform the decision.

&.



#####################################################################################
This e-mail message has been scanned for Viruses and Content and cleared 
by MailMarshal.
The Blackpool Sixth Form College.
#####################################################################################