LPEG tweaks

[Tony Finch <dot@...>]

I have a few tweaks to LPEG (mainly to re.lua) which I'd like to share.
There's a patch at the end of this message.

I've changed the identifier syntax to allow underscores, which is
compatible with Bryan Ford's standard PEG syntax.

I've added a .. operator to make it easy to search for matches. I chose it
so that raw lpeg can support a similar feature by overloading Lua's ..
operator. However the re.lua version is actually a prefix operator so you
can just write .. <pat> instead of "" .. <pat> to match without a left
anchor.

I allow bare identifiers in the right hand side of grammar definitions.
They are equivalent to %identifier if the identifier has an entry in the
definitions table passed to re.compile(), otherwise they are equivalent to
<identifier>. It also supports the lpeg.locale() predefined patterns but
not re.lua's short %a versions because that would interfere with using
many single-letter grammer non-terminals.

I like the following tweaks to the lpeg metatable:

	local lpeg_mt = getmetatable(lpeg.P(0))

	lpeg_mt.__mod = lpeg.Cf

	function lpeg_mt.__concat (a,b)
		return a * lpeg.P{ b + 1 * lpeg.V(1) }
	end

Tony.
-- 
f.anthony.n.finch  <dot@dotat.at>  http://dotat.at/
GERMAN BIGHT HUMBER: SOUTHWEST 5 TO 7. MODERATE OR ROUGH. SQUALLY SHOWERS.
MODERATE OR GOOD.


--- lua-5.1.4/etc/re.lua	2008-10-11 20:21:05.000000000 +0000
+++ lua-5.1.4/etc/re.lua	2009-10-29 16:12:30.000000000 +0000
@@ -30,6 +30,11 @@
   error(msg, 2)
 end

+local function search (p)
+  -- shorter than (1 - p)^0 * p
+  return m.P{ p + 1 * m.V(1) }
+end
+
 local function mult (p, n)
   local np = m.P(true)
   while n >= 1 do
@@ -49,7 +54,7 @@

 local S = (m.S(" \t\n") + "--" * (any - m.S"\n")^0)^0

-local name = m.R("AZ", "az") * m.R("AZ", "az", "09")^0
+local name = m.R("AZ", "az", "__") * m.R("AZ", "az", "__", "09")^0

 local exp_follow = m.P"/" + ")" + "}" + ":}" + "~}" + name + -1

@@ -71,6 +76,10 @@
   return cat
 end

+local BareID = Identifier / function (n,Defs)
+  return (Defs and Defs[n]) or (#n > 2 and Predef[n]) or m.V(n)
+end
+
 local Range = m.Cs(any * (m.P"-"/"") * (any - "]")) / m.R

 local item = Cat + Range + m.C(any)
@@ -102,6 +111,7 @@
         * (#exp_follow + patt_error);
   Prefix = "&" * S * m.V"Prefix" / mt.__len
          + "!" * S * m.V"Prefix" / mt.__unm
+         + ".." * S * m.V"Prefix" / search
          + m.V"Suffix";
   Suffix = m.Cf(m.V"Primary" * S *
           ( ( m.P"+" * m.Cc(1, mt.__pow)
@@ -128,7 +138,8 @@
             + "{~" * m.V"Exp" * "~}" / m.Cs
             + "{" * m.V"Exp" * "}" / m.C
             + m.P"." * m.Cc(any)
-            + "<" * name * ">" / m.V;
+            + "<" * name * ">" / m.V
+            + BareID * ("" - S * "<-");
   Definition = Identifier * S * '<-' * m.V"Exp";
   Grammar = m.Cf(m.V"Definition" / firstdef * m.Cg(m.V"Definition")^0, adddef) /
                 m.P