Re: Lpeg coroutine?

[Paul K <paul@...>]

Hi Gavin,

> A stream of characters produces a stream of strings, each of which is a prefix of its successor. To retest each such string against the same pattern seems a bit uneconomic. One wants the matching process to yield the state of the matching engine, so that the match can be resumed for the next character to be received.

There was a discussion last year on yieldable/streaming LPeg
(http://lua-users.org/lists/lua-l/2014-09/msg00018.html) as I've been
trying to do something similar to what you are doing.

My idea was to yield from a capture function, which (obviously) didn't
work because of the "attempt to yield across C-call boundary" error
(same in Lua 5.2, I haven't tried Lua 5.3). LPegLJ solves this problem
by being pure-Lua and I haven't tried Coco (http://coco.luajit.org/)
since it's Lua 5.1 only as far as I understand.

After looking at the structure of the lpeg state [1], I keep going
back to the original suggestion of having a yieldable operation in
lpeg. This may be achieved with a match-time capture similar to Cmt
(Cymt?), but that will "yield" from match when it gets "false" return
value.

The capture would then return the state object (probably as the result
of match() call), which could then be passed back to match() instead
of the pattern (as the state with its stack information is tied to the
pattern, it doesn't make sense to change the pattern anyway).

 This "capturing the state" may be used in other circumstances: I may
want to capture the state at the beginning of a block and then use
that state to "restart" the matching from that position, possibly with
a different subject string (starting from that position). Adding
"Cymt(true, function() return somecondition end)" would make parsing
fully resumable (all the other captures would be returned only in the
case of a "normal" match) and adding "Cy(true)" would allow
intermediate states to be returned as captures.

Paul.

[1] http://www.inf.puc-rio.br/~roberto/docs/peg.pdf