Re: how to (slightly) modify the Lua parser

[Mark Hamburg <mark@...>]

On Oct 12, 2009, at 9:08 AM, Francesco Abbate wrote:

> Hi all,
>
> I know that in this list there are a lot of clever guys so I guess  
> it is the good place to ask *very* difficult questions about Lua :-)
>
> For my project, GSL shell, I would like to slightly modify the Lua  
> parser to add some syntactic sugar. For example I would like to write:
>
> |x| sin(x)/x
>
> or
>
> |x, y| sin(x) * cos(y)
>
> as a function literal instead of
>
> function(x) return sin(x)/x end
> or
> function(x,y) return sin(x) * cos(y) end
>
> Incidentally, this syntax is supported by metalua.
>
> Otherwise I would like to index object with more comma-separated  
> values because for a matrix is natural to write something like m[i,  
> j] to refer to an element of the matrix. Right now I'm doing m:get 
> (i, j) but I would like to simplify the notation.
>
> It seems that metalua can offer all these things but I'm wondering  
> if it is really the good choice. The other alternative could be to  
> hack the Lua code to modify the parser. I'm ready to fiddle my hands  
> in the Lua code but the source code is quite complicated and I  
> didn't find any tutorial on 'hacking Lua' so it is very difficult to  
> start.
>
> Thank you in advance for any help.

I'm afraid I'm not going to answer your question though the general  
answer probably is of middling difficulty (you need to modify the  
lexer to recognize the bar token and you'll need to define a new case  
for expressions -- I would probably look at where the function keyword  
is accepted). But before you dive in, you probably need to consider a  
variety of questions:

* What sort of precedence does | <args> | <expression> have? In  
particular, how does it relate to commas? For example, is:

	| x | x + 1, 2

Equivalent to:

	function( x ) return x + 1 end, 2

Or is it equivalent to:

	function( x ) return x + 1, 2 end

Does this really do what you want in the context of expression lists?  
In other contexts?

* Do you also want to consider a syntax like:

	| <args> | do <statements> end

This would allow for more than just expression one liners.

* On a similar front, how about:

	| <args> | <statement>

For the cases where one has just one statement. That has the problem  
that you need a way to tell a statement (which won't need an implicit  
return) from an expression (which will).

* Do you want to treat these expression like tables and strings and  
allow them to be invoked without parentheses? This is probably  
particularly interesting for method calls:

	myCollection:map | x | sin( x ) / x

If so, should this be allowed for the heavy syntax as well:

	myCollection:map function( x ) return sin( x ) / x end

Of course, the further one pushes on these sort of constructs, the  
closer one gets to Smalltalk and Ruby blocks but note that those have  
interesting rules about returns v non-local exits. (A block can either  
return a value to its caller or can return/exit from the method in  
which it is created.) Non-local exit is an interesting semantic  
extension to Lua, but it isn't clear what it would mean for pcall.

But pulling back from the semantic question, there are presumably  
answers to some of these points available in Metalua. One could also  
resolve some of the questions through alternative syntax choices that  
provided an explicit end marker for the "syntactically lightweight"  
closure. This might also make dropping parentheses more appealing:

	myCollection:map<< | x | sin( x ) / x >>

(I'm not particularly attached to the above syntax.)

Mark