Re: return value of funcion used by lpeg.Cmt

[Roberto Ierusalimschy <roberto@...>]

> -- count lines
> 
> require 'lpeg'
> 
> local N = 0
> local function fline()
>     N = N + 1
>     return true
> end
> 
> local crlf  = lpeg.P'\n' + lpeg.P'\r\n'
> local line  = -1 + lpeg.Cmt( ( 1 - crlf )^0, fline )
> local lines = line * ( crlf * line )^0
> 
> lpeg.match( lines, io.open( arg[ 1 ] ):read( '*a' ) )
> io.stdout:write( N, ' lines\n' )
> 
> Clearly, the captured data of lpeg.Cmt has to be in memory as it is
> passed to fline(). But can I process a multi million line file this
> way or are all the multi million captures kept until lpeg.match() has
> returned? It was my hope that the 'clue' of lpeg.Cmt is that I can do
> that.

Cmt is processed in real time, so your reasoning is correct. Cmt
captures are 'dischared' as soon as the function executes, only the
resulting captures created by Cmt (if any) are kept.


> Ok, I still read in the whole file at once, so my memory has to be
> that big, at least...

Yes, and the extra memory for all those captures may be cheaper than
all these function calls (assuming you have virtual memory...).

-- Roberto