Status of token filters?

["Mark Meijer" <meijer78@...>]

Hi all,

In the spirit of recent discussions regarding Lua's flexibility and
its syntax/meta features, I was wondering what the current status is
on the topic of making token filters an official Lua feature. I
searched the mailing list archive and found posts mentioning token
filters throughout, up to three years back.

I understand that it was (quite sensibly) decided to first keep it as
an add-on and see what happens with it. A couple of nice experiments
were done with it, and there have been some interesting discussions
about it on this list (yes I've read about the potential can of worms
that token filters can bring, as well as the limitations, and possible
alternatives), but I don't really know what people have actually done
with it since, or if the idea has developed in any way.

However I do believe that many patches/tools/libraries are still
coming out (or being maintained) that could probably have been
implemented using token filtering (or alternative) add-ons, but
weren't. Has this idea been largely ignored after all? Is it time to
re-open this topic for discussion? Any comments at all?

Regards,
Mark