Re: how to (slightly) modify the Lua parser

[Francesco Abbate <francesco.bbt@...>]

Hi guys,

thank you very much for the suggestion. Actually I was able to make a patch for the 'compact function literal' and it works like a charm :-)

I was impressed how simple and well written is the Lua parser but that's not a surprise given the general level of quality of Lua.

About the priority, my idea was that it should act very much like a classic function literal. In this spirit I've given a priority 0 and I've thus used expr(ls, e) in the parser instead of subexpr(ls, v, priority). Note that the comma is not treated like an operator so the statement:

f, g = |x| 2*x, |x| 3*x

perform the assignment for f and g as expected.

Otherwise I don't want to heavily modify the syntax to support function literals without parenthesis as you suggested. I don't want to make another Ruby implementation :-)

I'm sending the patch for the interested ones. I guess it qualifies to be listed among the PowerPatchs.

Francesco

--- BEGIN OF FILE ---

--- lua-5.1.4/src/lparser.c 2007-12-28 16:32:23.000000000 +0100

+++ lua-5.1.4-real-mod/src/lparser.c 2009-10-12 20:08:06.000000000 +0200

@@ -592,8 +592,26 @@

   pushclosure(ls, &new_fs, e);

 }

 

 

+static void simplebody (LexState *ls, expdesc *e, int line) {

+  /* simplebody ->  parlist `|' expr END */

+  FuncState new_fs;

+  expdesc ebody;

+  int reg;

+  open_func(ls, &new_fs);

+  new_fs.f->linedefined = line;

+  parlist(ls);

+  checknext(ls, '|');

+  expr(ls, &ebody);

+  reg = luaK_exp2anyreg(&new_fs, &ebody);

+  luaK_ret(&new_fs, reg, 1);

+  new_fs.f->lastlinedefined = ls->linenumber;

+  close_func(ls);

+  pushclosure(ls, &new_fs, e);

+}

+

+

 static int explist1 (LexState *ls, expdesc *v) {

   /* explist1 -> expr { `,' expr } */

   int n = 1;  /* at least one _expression_ */

   expr(ls, v);

@@ -765,8 +783,13 @@

       luaX_next(ls);

       body(ls, v, 0, ls->linenumber);

       return;

     }

+    case '|': {

+      luaX_next(ls);

+      simplebody(ls, v, ls->linenumber);

+      return;

+    }

     default: {

       primaryexp(ls, v);

       return;

     }

--- END OF FILE ---

2009/10/12 Mark Hamburg <mark@grubmah.com>

On Oct 12, 2009, at 9:08 AM, Francesco Abbate wrote:

Hi all,

I know that in this list there are a lot of clever guys so I guess it is the good place to ask *very* difficult questions about Lua :-)

For my project, GSL shell, I would like to slightly modify the Lua parser to add some syntactic sugar. For example I would like to write:

|x| sin(x)/x

or

|x, y| sin(x) * cos(y)

as a function literal instead of

function(x) return sin(x)/x end
or
function(x,y) return sin(x) * cos(y) end

Incidentally, this syntax is supported by metalua.

Otherwise I would like to index object with more comma-separated values because for a matrix is natural to write something like m[i, j] to refer to an element of the matrix. Right now I'm doing m:get(i, j) but I would like to simplify the notation.

It seems that metalua can offer all these things but I'm wondering if it is really the good choice. The other alternative could be to hack the Lua code to modify the parser. I'm ready to fiddle my hands in the Lua code but the source code is quite complicated and I didn't find any tutorial on 'hacking Lua' so it is very difficult to start.

Thank you in advance for any help.

I'm afraid I'm not going to answer your question though the general answer probably is of middling difficulty (you need to modify the lexer to recognize the bar token and you'll need to define a new case for expressions -- I would probably look at where the function keyword is accepted). But before you dive in, you probably need to consider a variety of questions:

* What sort of precedence does | <args> | <_expression_> have? In particular, how does it relate to commas? For example, is:

       | x | x + 1, 2

Equivalent to:

       function( x ) return x + 1 end, 2

Or is it equivalent to:

       function( x ) return x + 1, 2 end

Does this really do what you want in the context of _expression_ lists? In other contexts?

* Do you also want to consider a syntax like:

       | <args> | do <statements> end

This would allow for more than just _expression_ one liners.

* On a similar front, how about:

       | <args> | <statement>

For the cases where one has just one statement. That has the problem that you need a way to tell a statement (which won't need an implicit return) from an _expression_ (which will).

* Do you want to treat these _expression_ like tables and strings and allow them to be invoked without parentheses? This is probably particularly interesting for method calls:

       myCollection:map | x | sin( x ) / x

If so, should this be allowed for the heavy syntax as well:

       myCollection:map function( x ) return sin( x ) / x end

Of course, the further one pushes on these sort of constructs, the closer one gets to Smalltalk and Ruby blocks but note that those have interesting rules about returns v non-local exits. (A block can either return a value to its caller or can return/exit from the method in which it is created.) Non-local exit is an interesting semantic extension to Lua, but it isn't clear what it would mean for pcall.

But pulling back from the semantic question, there are presumably answers to some of these points available in Metalua. One could also resolve some of the questions through alternative syntax choices that provided an explicit end marker for the "syntactically lightweight" closure. This might also make dropping parentheses more appealing:

       myCollection:map<< | x | sin( x ) / x >>

(I'm not particularly attached to the above syntax.)

Mark