Re: Yieldable LPEG Parser

[William Ahern <william@...>]

On Wed, Feb 01, 2012 at 04:27:16PM -0800, William Ahern wrote:
> On Wed, Feb 01, 2012 at 03:51:41PM -0200, Roberto Ierusalimschy wrote:
> > > [Hmm, that brings to mind another question:  How much of the input
> > > string does it have to accumulate in memory?  Can it start discarding
> > > earlier portions of it at some point?  If not, it wouldn't be so useful
> > > for the use I have in mind: parsing really big files without needing to
> > > have them entirely in memory.]
> > 
> > That is an important point. Also, do you have benchmarks comparing your
> > patch to standard LPeg?
> > 
> 
> Attached is my LPeg JSON library used for testing.

Same library, but parsing a single 5.1MB JSON file:

% for M in json json-10-2; do for I in 1 2 3; do \
time ./rfc-index.lua ${M} < /tmp/rfc-index.json; \
done; done;
lpeg 0.10 (yieldable)
./rfc-index.lua ${M} < /tmp/rfc-index.json  2.22s user 0.45s system 98% cpu
2.699 total
lpeg 0.10 (yieldable)
./rfc-index.lua ${M} < /tmp/rfc-index.json  2.22s user 0.45s system 99% cpu
2.692 total
lpeg 0.10 (yieldable)
./rfc-index.lua ${M} < /tmp/rfc-index.json  2.22s user 0.45s system 99% cpu
2.690 total
lpeg 0.10
./rfc-index.lua ${M} < /tmp/rfc-index.json  2.15s user 0.45s system 99% cpu
2.621 total
lpeg 0.10
./rfc-index.lua ${M} < /tmp/rfc-index.json  2.15s user 0.45s system 99% cpu
2.615 total
lpeg 0.10
./rfc-index.lua ${M} < /tmp/rfc-index.json  2.15s user 0.44s system 98% cpu
2.622 total