Re: Yieldable/streaming LPEG

[William Ahern <william@...>]

On Mon, Sep 01, 2014 at 11:30:25AM +0200, Dirk Laurie wrote:
> 2014-09-01 9:55 GMT+02:00 Sean Conner <sean@conman.org>:
> 
> > I often wished this were true of LPeg:
> >
> >         lpeg.match(pattern,subject[,init])
> >
> >                 pattern - LPeg expressoin
> >                 subject - if string, use as input (what we have now)
> >                           if function, call function to get input
> >                           (much like how load() works)
> >                 init    - where in input to start
> 
> At present LPeg can diagnose that a pattern is left recursive
> and bail out instead of looping infinitely. If LPeg did the above,
> it could in principle go into an infinite loop. On the other hand,
> no more so than can load().
> 
> How hard would it be, having the code of load() as a model,
> to patch LPeg so that it behaves as you describe?

I started a patch here a couple of years ago

	http://25thandclement.com/~william/projects/lpegk.html

which hacked LPeg to accept a vector (e.g. a load-like function), and which
supports yielding. This allows the subject pattern to be a different block
of memory as it grows. However, it doesn't address the backtracking issue.