Lpeg and deep nesting

[Luís Eduardo Jason Santos <jasonsantos@...>]

Hi, list

I was writting an LPEG parser and when I got it all nice and shiny I've 
decided to test it on a real world example.
As expected, it blew up on some cases. Studying those cases I pinpointed 
the problem situation, and wrote a simplified example as follows:

<code>
require'lpeg'

local WS = lpeg.S" \t\r\n"^0
local T = lpeg.P{ lpeg.V"content";
  content = (lpeg.V"token" + lpeg.V"text")0,
  text = (1-lpeg.S"[]")1,
  token=WS * ("[" * lpeg.V"content" * "]") / print * WS
}

T:match[==[
  [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [
  [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [
  [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [
  [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [
  [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [
  [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [ [
      content
  ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ]
  ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ]
  ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ]
  ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ]
  ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ]
  ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ] ]
]==]
</code>

This example raises a "too many pending calls/choices" error.

I imagined Lpeg stores on stack some sort of closure for the inner 
captures it might find, and reading the source I've noticed the 400 
entries limitation.

My questions are: did I understand it right? if so, is there a recipe to 
circumvent this limitation?

Att.
Luís Eduardo Jason Santos