Re: future of annotations in Lua?

[Hisham <h@...>]

On Fri, 7 Jun 2019 at 11:11, Roberto Ierusalimschy
<roberto@inf.puc-rio.br> wrote:
>
> > [...]
> > that using '<' like that would introduce ambiguites that would
> > preclude general function annotations in the commonly-adopted
> > annotation style. The following would be hard to parse without
> > significant lookahead:
> >
> >    local x = y
> >
> >    <my_custom_annotation(with, arguments)>
> >    function my_global_function()
> >       -- ...
> >    end
> >
> > If the door is to be left open for such a generalized mechanism in
> > future versions of Lua beyond 5.4, then adopting a syntax based on an
> > unused token such as @annotation would be advisable. Matching the
> > annotation syntax of other languages would be a side benefit.
>
> There is a big difference between all those syntaxes and the one
> we are using in Lua, not because of the peculiarities of '<name>'
> versus '@name': They are all prefixed to the whole item to which they
> apply. Following their syntax, we should write '@toclose local x = 1',
> instead of 'local @toclose x = 1'.

The "whole item" the annotation is applying to is the variable, and to
my eyes that is `x`, and not `local x`, because a `local` declaration
may have multiple variables. Of course, given Lua's general
support for multiple returns and expression lists, I was expecting
something like

   local x, <toclose> y = 0, io.open("hello")

to be valid, but looks like it isn't. (I understand it can be worked around
with an extra line, but it seems like a needless inconsistency, unless I'm
missing something.)

> For locals it is not a big difference, but the following looks weird to
> my eyes, no matter the details:
>
>      @my_custom_annotation(with, arguments)
>      function my_global_function()
>         -- ...
>      end

As of 2019 this idiom with the linebreak is pretty standard-looking
syntax for languages that have annotations. It did look weird to me
as well the first time I had to write programs with Java annotations
years ago, but like all things syntax, it eventually became natural
with use, and I was unsurprised to see other languages adopting it.
I believe it has become an idiom to stack annotations in function
declarations with line breaks like this because otherwise the
function declarations grow too long. (In realistic examples, of course:
that's why I stopped using examples like `@a function f()` or
`t[t+1]` when discussing syntax -- what looks tidy in a discussion
with single-letter variable names breaks down in real world
programs with longer variables.)

For local functions, we have an interesting situation.
I assume for locals the direct Lua adaptation for that would become

   local @my_custom_annotation(with, arguments)
   function my_local_function()
      -- ...
   end

which could end up looking like a local variable followed by a global function
to the uninitiated. Allowing the annotation to be prefixed to the
entire `local` declaration
(made possible by the unambiguous @ syntax), one could do the more natural

   @my_custom_annotation(with, arguments)
   local function my_local_function()
      -- ...
   end

Given that `local function f` is already a syntactic shorthand to
`local f = function`, I guess `@annotation local function f` could be
a syntactic shorthand to `local @annotation f = function`.

This would preserve the notion that annotations apply to the variable
name being declared, which brings back to the point about variable
lists. To reiterate on the issue of annotations in variable lists
(which is a matter of concern for the current applicability of the
upcoming Lua 5.4 features rather than musing about a future
extensibility), the following scenario seems like an obvious and
common application for toclose, if annotations were allowed in
multi-var lists:

   local @toclose connection, err = make_connection(host, port)
   if not connection then
      return nil, err
   end

Requiring people to do

   local connection, err = make_connection(host, port)
   if not connection then
      return nil, err
   end
   local @toclose connection = connection

every time seems really unfortunate. (That's another tricky thing in
syntax discussions: people's tiny examples often omit error checking,
which real world programs need to do. Lots of examples were floated
around in other threads with people doing `local <toclose> f =
io.open("file.txt")`. The moment I try to catch the err in that line,
the pretty example breaks down and we end up with a feature that can't
really be used in practice without a verbose idiom.)

> (BTW, note that
> the C++ [syntax] in Lua would have the same problem that <name>,
> assuming prefixed attributes.)

(C#, but still) yes, I do realize and that's why I did not suggest it.

So, I guess the short version of my rambling is:

1. please consider @ rather than <> in Lua 5.4 if you consider
extending this someday in Lua > 5.4
2. please allow per-variable annotations in multi-variable
declarations in Lua 5.4, no matter what syntax you end up choosing.

-- Hisham