Re: Are Lua's own lexer/parser suitable for writing a syntax highlighter?

[Steven Degutis <sbdegutis@...>]

> For syntax highlighting, you don't need the parser, just the lexer, right?

I was under the impression that the parser just gives me more
information about the file, so that I can color things more
specifically, e.g. it may recognize an identifier as being a local
variable or a global, so I can color them differently.

But perhaps this is not the case. I have not heavily looked into the
source code of Lua's parser or lexer yet.

> The only interface that should interest you is llex and the SemInfo
> struct. To see how these work, search the archives for "proxy.c" and
> "token filter". See for instance
>         http://lua-users.org/lists/lua-l/2012-03/msg00309.html
>
> See also my recent rewrite of my lstrip as a token filter:
>         http://lua-users.org/lists/lua-l/2014-07/msg00058.html
>
> The code in lstrip should be a good place to start.

Thank you! I will read your references.

> The only problem is that you won't be able to run Lua programs
> since the token filter in lstrip eats all tokens and the parser sees
> an empty stream.
>
> You may use a global variable to switch off the token filter and then
> allow your app to run Lua programs. This should be easy to do.

Would creating a new private lua_State for my syntax-highlighter's be
a reasonable way to work around this?

-Steven