Language design is intrinsically hard, it's bound to have an outrageous learning curve, and syntax extensions are a simplified case of language design. There can be two partial fixes to this:
- optimize the whole language for syntax flexibility, i.e. reinvent Lisp yet another time.
- or favor extensions reuse, so that if it's hard, at least it doesn't have to be done more than once.
Making languages successful in the real world is all about the law of
association, about critical mass.
Indeed, there's a risk of balkanization of the language if it supports easy syntax extensions, and it might well be what plagued Lisps as development platforms. The existence of the official Lua definition and implementation by Lualabs is extremely important with this respect: they act as a de facto standardization body, and it demands extreme cautiousness before they add anything to the language.
Successful languages eventually crumble under the support of unfortunate design choices, that have been made too lightly. So the rationalized argument for rejecting most proposals, including the one which sparked this discussion, is: "
the gain is marginal at best, whereas touching the standard has huge and unpredictable consequences. Risks are way out of proportion to the achievable gains". So the fact that such tiny proposals are silently ignored by the Lua team is both very reasonable, and completely unrelated to the existence of metaprog systems. These systems don't rob you from the Lua team's attention, they just give you a limited ability to do without it.
Also, the current existence of several metaprog systems, sometimes without a clear purpose beyond the NIH syndrome, certainly doesn't help with balkanization, but that will eventually settle out.
A static meta-programming system offers compromises different from sticking to the plain language:
- it gives up the standard's conservatism. You lose some interoperability, but you get more cutting-edge features. That's the same kind of choice you make when picking a stable or experimental Linux distribution.
- it can still retain a good deal of interoperability, if extensions can be written and used modularly. You can restrict yourself to mature extensions that are worth it, just as you can restrict yourself to good quality 3rd party libraries.
- it gives their chance to syntax experiments. As written above, the sane attitude for the Lua team is to reject virtually all proposals, especially the superficial ones. The only way to prove that your extension is one of the <1% that are worth it is to field-test it on significant code bases, before it is even remotely considered for inclusion in plain Lua. For that, you need something intermediate between inclusion in the official standard, and a shady patch of the original distro. That's what a proper metaprog system provides. Most community-suggested improvements first existed for years as patches, before getting the Lualab blessing.
Static metaprog makes such language changes more flexible. If designed correctly, the system must also make them more robust, let them compose better, and still carry a (more relaxed) sense of standardness.
