Re: [ANN] metalua 0.4

[Fabien <fleutot+lua@...>]

What does lisp syntax have to do with writing reusable libraries?

I agree that Java has an awesome set of libraries, and even better, a
library-writing culture, but this seems to have much more to do with the
way the language was "managed" than anything about syntax...

(The following views are only mine. Obviously, they shape metalua's design).

A language is an interface between programmers and hardware, so it has social/psychological/pedagogical features which are just as important as its formal properties. If a language can't be efficiently ported on regular hardware, it's the language that sucks, not the hardware. Similarly, if it doesn't interface properly with its communities of coders (fails to build up standard coding practices, good libraries, trust...), the language sucks, not the people. Ergo Lisp sucks. Many Lisp zealots dismiss the language's failures as "merely social", but that's missing the purpose of a language entirely: failing socially is just as bad as failing technically.


Now, if you agree that there must be something fundamental, in lisps designs, that drives them all to social failure, what could be the root causes? I don't think the scare factor induced by parentheses is a real reason. Just look at the kind of scary, steaming piles of crapware Java developers are willing to learn: they're OK with filling a 2 meters long shelf with manuals explaining how to have ORM and hibernation. The more verbose and unreadable a design pattern is, the more they love it. XML is much much scarier than sexps, yet consulting services are hysterically excited by it. No, really, it would take much more than parentheses to scare a Java programmer.

So what could be the features of "normal" syntax that are missed by sexps? Most notably, normal syntax gives you a feeling of what's idiomatic vs. what's weird. This means the syntax encourages you to write in a certain way, and hopefully everyone will be encouraged to write in the same style, thus understanding each other's code better. This helps bringing you interoperability and favors the writing of reusable code.

Let's take a concrete, Lua-centric example: about once a month, there is a discussion on the mailing list about "fixing" the anonymous function syntax. What drives people so crazy about it? It's not the couple of extra keystrokes, nor the screen real estate eaten by "(function() end)"; it's the fact that the current syntax carries a message they strongly disagree with, which says: "anonymous functions are not a lightweight feature to be used pervasively, passing them as function parameters is not the Lua Way, that's why it looks syntactically odd. If you use them, it ought to be because you do something exceptional, so your code should have a somewhat exceptional appearance". They feel wrong when they use it, and they think they ought to feel good. Now I don't think it was intended, in the design of the Lua syntax, to convey this message, but that's what programmers perceive in practice. Like kids, languages tend to get a life and tastes of their own, that are not completely controlled by their creators :)

With sexps, it's much harder to create, maintain and convey such opinions in the code's appearance. Everything tends to look casual, and if it doesn't, just add a macro to make it look OK. It's difficult to build a shared seense of Good and Evil under these conditions, and without this consensus you'll have a hard time building a functional social group!

Then, there are macros. Macros are extremely powerful; as we've seen, they can make everything look OK, so it's hard for the language to convey opinion, and for a consensus on what's idiomatic to emerge. Another problem is, since by using them you essentially design your own single-use language for your application, you're the only one in the world using that language variant. So long for integrating other people's libraries, and for other people to integrate your libraries. We know how to package functions and data structures in modular, reusable ways; we don't know how to do that with macros, so macros hurt code reuse.

Metalua tries to address these points: first, being mostly Lua, it's got a clean syntax, with what I think has a good compromise between directivity and freedom of _expression_. And wrt macros, it tries to convey a different idea about what kinds of macros are OK. Macros have a big cost in terms of modularity and maintainability. You're expected to pause and ask yourself "is it worth it?" before writing one, and when you do, you're expected to do it properly, in a distinct extension file. You can do otherwise, but if you do, your code will look strange, so you'll have been warned.

In metalua:

* it's easier and it looks cleaner to put extension code in a separate file, which itself preferably goes in a separate "extension" directory. That's an incentive to be modular and to write reusable stuff. Plus, when you're about to write some macro code, you'll have to switch to another file : it takes a second you can use to think, and it will feel odd if your macro has no reasonably reusable purpose.

* it's easier and looks cleaner to reuse the combinators provided by gg and mlp, rather than inventing your own. So unless you have a very special issue, you'll naturally use the tools that created the Lua parser; chances are better that your syntax will be more Lua-esque. If you're about to do something cheesy, you'll know because you'll find yourself writing a new combinator.

* meta-operators +{...} and -{...} visually stand out in the code, and that's a feature: if you need them, you're certainly about to do something that deserve the reader's attension!

These "proper ways" are easy to circumvent, but if you do it, it's on purpose: you know you're doing dangerous things, and hopefully you'll refrain from doing it gratuitously.