Re: Why was 'const' implemented in Lua 5.4?

On Mon, May 30, 2022 at 2:56 PM Adam Higerd <chighland@gmail.com> wrote:

On Mon, May 30, 2022 at 1:41 PM Ryan Starrett <pyryanggg@gmail.com> wrote:
On Mon, May 30, 2022 at 2:37 PM Adam Higerd <chighland@gmail.com> wrote:
On Mon, May 30, 2022 at 1:17 PM Ryan Starrett <pyryanggg@gmail.com> wrote:
On Mon, May 30, 2022 at 2:11 PM Adam Higerd <chighland@gmail.com> wrote:
On Mon, May 30, 2022 at 1:04 PM Ryan Starrett <pyryanggg@gmail.com> wrote:
This has been a small debate in a couple of my Lua circles for a little while. I've never quite understood the thoughts behind this new feature. Today, I took some preliminary observations into the bytecode behind 'const' and non-constant counter-parts.

What I learned when doing this, is that constant locals which are associated with immutable values will behave more like a preprocessor macro than a position on the stack.

For example, printing a constant local defined as "my number" then printing it will produce the following bytecode:
0+ params, 2 slots, 1 upvalue, 0 locals, 2 constants, 0 functions
1 [1] VARARGPREP 0
2 [2] GETTABUP 0 0 0 ; _ENV "print"
3 [2] LOADK 1 1 ; "my number"
4 [2] CALL 0 2 1 ; 1 in 0 out
5 [2] RETURN 0 1 1 ; 0 out
Which is absent from the MOVE instruction from the non-const counter-part, so LOADK puts the value of the constant local directly into the instances of the local, instead of MOVE taking the value from the local.

Conversely, when a mutable value like a table is associated with a constant local, there are no bytecode differences, which means that a const local for a table is essentially identical to a non-const local, minus compile-time semantics like reassignment.

I know Lua is typically constrained on features because it wishes to preserve small binary sizes, but the const feature seems redundant with my current understanding. At this point, I don't know why it was added. Why was it added?

The rationale is reasonably well-documented in the mailing list archives: the underlying behavior was necessary to implement to-be-closed variables, so it didn't add overhead to expose <const> as a feature in the parser. Some developers like having the restriction on reassignment (it's the same semantics as JS const) so it's very much a "why not?" kind of thing.

/s/ Adam

> Why do to-be-closed variables need constness enforced?

I'm working from a distant memory here so I could have the details wrong a little bit, but if I recall correctly:

A to-be-closed variable creates a scope, and when that scope ends, it needs to call the __close metamethod unless the variable contains nil or false. Trying to put an object that can't be closed into a to-be-closed variable is an error. It would be far more complex to check to see if every assignment to a to-be-closed variable would be an error or would make it nil/false and therefore change the later behavior of the function. Making the variable not reassignable means that you can just do a simple check at parse time instead of having to add a code path to the interpreter. And then as a semantic issue, even if you were to support reassignment, what happens to the value that WAS there before? Should it be closed immediately because the value was removed, or should it not be closed because the variable is still in scope?

Since there could be a number of different code paths leading out of that scope (`break`, `goto`, `return`, and throwing an error can all do it), it works a lot better to figure out what needs to be done up front. Otherwise every possible exit from the scope has to check to see if there's something to be done there. And given the choice between doing the check in one place when you KNOW that a to-be-closed variable is involved, and doing the check in an arbitrary number of places where you'd have to go out of your way to keep track of whether or not there's a to-be-closed variable in scope... the first one means you don't have to add complexity or overhead if you aren't using the feature.

Finally, just as a matter of intent: The use case of to-be-closed variables is for things like cleaning up open files. It seems pretty unlikely that you're going to WANT to switch that out in the middle of the scope.

If you have a good use case for reassigning a to-be-closed variable, then all you need is a wrapper table that delegates its own metamethods to an inner object that you can reassign, and then you can choose what behavior you want in the implementation of those metamethods. But if to-be-closed variables are reassignable, then there's no good workaround to go the other direction.

/s/ Adam

> This explanation of why constness was necessary is very insightful and I understand the dynamics here much better currently. Thanks.

Happy to help. :)

/s/ Adam

(P.S. Top-posting is poor netiquette.)