Re: Lpeg and Ct w/ nils

[Peter Odding <xolox@...>]

Thomas Harning Jr. wrote:
> I'm working on a JSON parser using lpeg and have it mostly working.
> 
> One corner case I've run into is where two value types in JSON (null
> and undefined) are being mapped to nil captures.
> 
> Some problems however:
> 1) lpeg.Cc(nil) does not seem to work right, it causes a function to
> be put in the chain somewhere.  I worked around it by using  '/
> function() return nil end'  Though this seems suboptimal.

Are these nil's directly returned to the caller of your parser? If not, you
could use a reference type value as a sort of sentinel, to be recognized (and
replaced?) after lpeg.match has done it's job. All it would need is:

   local _nil, _undef = {}, {}
   local patt = Ct( ... P'null' * Cc(_nil) + P'undefined' * Cc(_undef) ... )
   ...
   for i, match in pairs(matches) do
      if match == _nil or match == _undef then
         matches[i] = nil
      end
   end

or am I just replacing your empty function with an empty table? In any case I
think a special marker variable that is local to your parsers scope is more
generic than a function returning nil. It could also save some bytes, though I'm
not actually sure a function is bigger than a table if both are empty, or
whether this is relevant :).

> 2) lpeg.Ct( ... ) with nulls inside do not seem to increase the size
> of the array as expected.
> 
> Ex: [null, 1,2,3, null, 5] results in an array  w/ # = 4
> {nil, 1,2,3, nil, 5} results in an array with #= 6
> 
> My workaround was to use a 'function' value to represent nil, and then
> apply the function to the result of lpeg.Ct, iterating over the table
> cleaning out those nil representations.
> Before processing i grab #t and store it in t.n, after removing the
> values (in the above case and others) #t will no longer == t.n, so I
> leave in t.n for the length.  If #t == t.n, then i remove that
> element.
> 
> Any ideas on how to fix this up w/o my hacks.
> 

Even though it might feel like a hack there need not be anything wrong with some
 post-processing ;)

 - Peter Odding