Re: speed of Lua/LPeg parsers

[Mike Pall <mikelu-1009@...>]

Duncan Cross wrote:
> This reminds me of something I was wondering about: could a special
> version of LPeg be created that takes direct advantage of LuaJIT
> acceleration, so that the "bytecode" of the PEG patterns itself gets
> JITtered?

Roberto already answered about LPeg/JIT. But I should mention that
parsers have rather unique needs. It's hard to integrate them
efficiently into a JIT compiler targeted at an imperative language. 

When I get around to it, I may try to do something like that for
the built-in Lua pattern matching functions. Basically a trace
compiler for patterns. Not sure someone has done this yet and not
sure whether this will work out.

> Does LuaJIT support (even conceptually, if not at this
> precise point in the beta) external C modules that have
> JIT-specialised functions, like the functions in the standard Lua
> modules are?

I've been thinking about some kind of extension system. But for
that one needs more generality in the LuaJIT core, too (esp. the
IR and the backend).

> If so, would it be possible for a single dynamically
> linked library to contain both the standard and JIT versions, or would
> they need to be separate?

That's the easy part. Any such extension would need to register
itself with the JIT compiler. If the registry function is missing,
it's running under plain Lua and it should simply skip that step.

> Or is this all a blind alley, and the only
> realistic way would be a port of LPeg to Lua itself?

Well, there's a big impedance mismatch between these two. So this
would either be quite slow or you'd have to use lots of low-level
primitives on the Lua side (e.g. the planned struct/buffer support
of the FFI).

Not sure which is the better way to go.

--Mike