[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: Yieldable LPEG Parser
- From: Miles Bader <miles@...>
- Date: Thu, 02 Feb 2012 11:50:34 +0900
Tony Finch <firstname.lastname@example.org> writes:
> Miles Bader <email@example.com> wrote:
>> [Hmm, that brings to mind another question: How much of the input
>> string does it have to accumulate in memory? Can it start discarding
>> earlier portions of it at some point? If not, it wouldn't be so useful
>> for the use I have in mind: parsing really big files without needing to
>> have them entirely in memory.]
> lpeg doesn't bother trying to discard unneeded string prefixes. In theory
> it can only discard the prefix of a string that is not covered by any
> captures and which has no alternation backtracking points in it.
I suspect that in many cases, backtracking doesn't cover much of the
file, or the grammar can be arranged so that this is the case...
[I don't know how easy the implementation makes _detecting_ when parts
of the text are discardable though...]
As has been discussed elsewhere on this thread, the capture issue could
probably be sorted out, e.g. by lazy conversion of capture contents into
real Lua strings.
Pray, v. To ask that the laws of the universe be annulled in behalf of a
single petitioner confessedly unworthy.