David Manura

wiki

Wiki pages

There are Lua wiki pages and projects I've been involved in.

Code and Projects:

• LuaToCee - convert Lua to C code using C API calls
• InlineCee - simple module that allows C code to be embedded inline in Lua code.
• LuaDist - CMake-based ports system for Lua and non-Lua packages.
• LuaFish - Lua 5.1 parser, macro system, and static type checker in LPeg [1] (related to Metalua [2])
• LuaSearch - effort to standardize Lua module documenation via POD
• ModuleReview - effort to review/classify modules in terms of quality and usefulness for Lua distributions
• SlightlyLessSimpleLuaPreprocessor - enhanced version of the enhanced version of the Lua preprocessor
• LuaMatrix - pure Lua implementation of matrices
• LuaProxyDll - avoid DLL dependency issue
• LuaCom - Lua interface to Microsoft's Component Object Model (COM) - patching for 5.1
• DetectingUndefinedVariables - checkglobals function is an alternative to etc/strict.lua and luac -p -l - | lua test/globals.lua.
• LuaGenPlusPlus - A C++ header file to generate efficient sequences of Lua C API calls at compile-time via template metaprogramming techniques.
• LuaInterpreterInLua - reimplements the Lua 5.1 interpreter (lua.c) in Lua.
• ListComprehensions - pure Lua implementation of List comprehensions via dynamic code gen
• LuaBalanced - match delimited snippets of Lua code in a string
• CommandLineParsing - command line parsing (getopt)
• LuaPatch - Lua implementation of the patch utility
• StringLibraryInLua - partially reimplements Lua's string library
• ExPattern - regular-expression like support implemented in terms of Lua patterns and code generation
• ModuleCompressDeflateLua - DEFLATE (gzip/zlib) implemented in pure Lua
• SerialCommunication - serial communications in Lua/alien
• SourceOptimizer -- utility for optimizing Lua source, inlining functions
• StringQuery - string pattern matching and replacement library inspired somewhat by jQuery

General Topics, Patterns, Tricks, and Design:

• LuaNews - A summary of the latest happenings on the Lua mail list and elsewhere in the Lua world.
• LuaStyleGuide - every language has a style guide(s)
• LuaModuleFunctionCritiqued - please don't use the module function
• ModuleExecutionProposal - proposal+patch to add a new Lua command-line switch to load a function with a given package name and execute it as a script
• ModuleDefinition - various ways to define modules
• BuildingModules - (section on avoiding/mixing C runtimes)
• MetaLuaRecipes - Metalua syntax extensions
• ResourceAcquisitionIsInitialization - RAII is poorly supported in Lua
• StatementsInExpressions - relating to Lua statements cannot be used inside expressions without anonymous functions
• ExpressionsAsStatements - Lua expressions are not statements, though we can simulate the effect.
• VarargTheSecondClassCitizen - relating to varargs not being first class objects limits expression
• MutableFunctions - demonstration that closures can implement the semantics of Lua tables without using any Lua tables (functional programming)
• FunctionalTuples - a code generation approach to implementing tuples of length N
• LuaVersusPerl - Lua and Perl have different applications
• LunaticPython - call Python from Lua
• ErrorHandlingBetweenLuaAndCplusplus - issues in mixing Lua with C++ rather than C
• SourcePreprocessing - transforming source code before execution
• LuaDataFormats - Lua as a data representation language
• CppMetaprogramming - metaprogramming C++ not with templates or macros but with Lua.
• MakingLuaLikeRuby - Rubyesque Lua.
• GeneralizedPairsAndIpairs - why can't a table pretend to be another table?
• SandBoxes - issues relating to sandboxing: running untrusted Lua code in a restricted Lua environment
• LuaDesignPatterns - Lua design patterns
• LuaHacks - novel things you shouldn't do but could
• DecoratorsAndDocstrings - associate metadata or help with an object
• CustomOperators - simulating support for custom operators
• ImmutableObjects - topics of immutability in Lua
• StoringNilsInTables - topics relating to nil not being explicitly stored in tables

• ShortAnonymousFunctions - making function() ... end less verbose
• LuaTypeChecking - approaches to type checking in Lua, a dynamically typed language
• ReadOnlyTables - preventing writes to Lua tables.
• StringInterpolation - placing variables inline to strings, like Perl "x = $x"
• ColonForMethodCall - concerning the distinction in Lua of o:f() v.s. o.f()
• ObjectProperties - getters/setters that look like table field accesses
• LuaCarp - Perl Carp implemented in Lua (handle error level transparently)
• AssociativityOfConcatenation - discussions related to concatenation being right associative
• TernaryOperator - addressing the lack of a C-like ternary conditional operator (a?b:c) in Lua
• TableScope - solutions (and patch) to allow table constructs to enclose a scope
• CnumberPatch - fast implementation of getter/setter handler functions for locals - Lua patch.
• SimdExperiment - experimental patch to Lua VM to support a type of Single Instruction Multiple Data (SIMD)
• FileGlob - matching filenames with glob expressions
• PortableNetworkGraphicsParser - partial PNG graphics file parser (example)
• TheEssenceOfLoadingCode - explains the relationship between the various ways of loading external code in Lua.
• ExpressionTemplatesInLua - a Lua coding technique similar to C++ expression templates
• OutputLuaTableToHtmlFile - Lua table to HTML/CSS with expand/contract and xref links
• RuntimeSyntax - automates the run-time simulation of what resembles syntax extensions
• MethodChainingWrapper - cooperatively add custom methods to built-in types like strings and functions, particularly when using method chaining
• TablePreallocation - methods to preallocate tables from Lua
• SleepFunction - pause for N seconds
• GettingVariablesFromValues - pass-by-reference and the like for Lua
• StringTrim - comparison of string trim operations

Comments/Annotations on Other Documents:

• LuaReferenceManualComments
• ProgrammingInLuaComments
• LuaSource
• WikiStructure

Lua 5.2 Wishlist

These features seem likely to be included in 5.2 (LuaFiveTwo):

• Add BitwiseOperators to standard library
• Protect against triggering undefined behavior in C libraries from Lua standard libraries [6]
• __len metamethod and the like for tables (GeneralizedPairsAndIpairs) -- however, I'm not sure the accepted solution goes far enough in making Lua tables truly virtualizable (LuaVirtualization) and avoiding the get v.s. rawget distinction in API calls.

These seem simple to implement and not unreasonable, though there's no indication they have been accepted by the core team:

• Abstracted liolib.c [7]
• Clearer syntax errors [8][9][10]
• ModuleExecutionProposal
• Extend C API to support missing operators [11] -- e.g. lua_compare, lua_binop, and lua_unop.
• Add efficient tests for existence of operators on objects [12][13]
• Improved DetectingUndefinedVariables (possible with patch to debug.getinfo or lbci -- see "checkglobals")
• Fix "package.loaders" misnomer [14][15] (package.loaders contains searchers not loaders).
• Add more flexible and efficient form of lua_cpcall [16], supporting multiple arguments and return values and avoiding allocation. Add versions of lua_pushstring and others that don't longjmp [17]

These address common problems that don't have very good solutions in 5.1, though the proposed solutions may be slightly unclear or controversial:

• ShortAnonymousFunctions
• Cleanup on scope exit (ResourceAcquisitionIsInitialization)
• Make the module system more consistent - see LuaModuleFunctionCritiqued and LuaRocks efforts in (and criticisms of) redefining require.[3]
• Allow 'a:b()' to not depend on 'a.b' (e.g. __mcall or __mindex metamethod) [18]
• Overcome implementation limits in number of locals [19] and concatenation AssociativityOfConcatenation.
• The level parameter used in the error function lacks maintainability since a function might be called from different levels and those levels might be refactored. See LuaCarp for some effort to improve this.
• Manipulations with ... are somewhat awkward unless you copy it in a table (not as efficient) and handle nils correctly. See also VarargTheSecondClassCitizen. Something like the proposed #... and ...[i].

Some other interesting proposals that I might not even be fully convinced myself of:

• Allow pragmas in source to inject debugging info into the bytecode. [20] (useful for preprocessors of Lua source) (note: Metalua currently uses its own bytecode generator because of such limitations).

• Lua is in need of optional interfaces to non-ANSI C functionality (e.g. file directory listing) that are more accessible. ExtensionProposal and LuaRocks may help here. Real programs often need non-ANSI functionality, so why not standardize that functionality to the extent that it can be standardized?

• The inability to incorporate statements inside expressions limits the expressibility of Lua. Well, technically you can do this via an anonymous function, but allocating anonymous functions as such is not as speed efficient (nor is Lua's syntax for it as syntactically concise). This can be useful in metaprogramming, and Metalua recognizes this (see "Statements in expressions" in the Metalua Manual) and extends the Lua parser/compiler to allow it while requiring no change to the Lua VM. See also the "let" statements mentioned below on this page and "Stored Expressions" in LuaDesignPatterns.

• The highly dynamic nature of Lua limits its performance. True, Lua is fast, but it could be faster. For example, x*x is generally faster than x^2 since Lua cannot reduce the latter to the former at compile time (think: x might have a __pow metamethod). Macro processing could alleviate this (e.g. "transforming "y = SQUARE(x+y) + 1" into "y = (let t1 = x+y in t*t) + 1"). See also LuaJIT and LuaToCee. Global, or at least whole-file, flow analysis can help here.

• A Lua table cannot impersonate a file handle (or stream). For example, you may have a function that writes to a given file object. You might want to pass to that function not a real file object but an object with the same interface as a file object and having an implementation that does something else (e.g. appends to a string buffer). One use of this is would be to redirect standard output not to a file but rather to a text box in a GUI. This might be accomplished with new metamethods for file access (e.g. __read and __write).

• Fix the lua51.dll v.s. lua5.1.dll inconsistency in LuaBinaries / Win32. See LuaProxyDllTwo.

• for _,v in ipairs(t) is ugly, with the main complaint is that the meaning is not immediately apparent to users less familiar with Lua (Lua users can get used to it).

• Is there an efficient (O(1)) way to remove all elements from a table without changing the identity of the table (e.g. by allocating a new table)? Is this important? Would it be useful to provide a function in Lua that preallocates an array of size N (like lua_createtable)? [21]

• There is much need for a more standard way to document Lua modules. Perl has POD, and that was proposed in Lua in LuaSearch, but it hasn't yet catched on. Natural Docs has some benefits but requires some patches and has some questions of its own [4]. The benefits of these two documentation formats is that they are language neutral. (I have more to say on this.) Some want Python-like docstrings queryable at runtime though (see "Decorators, Annotators and Docstrings" in LuaDesignPatterns).

• There is a lack of consistency and centralization for Lua modules--in terms of locating, installing, documenting, testing, dependency tracking, etc. Think: Perl's CPAN. Others have noted this too [5]. [LuaForge] is part of the solution. [LuaRocks] may be part of the solution too. I believe that LuaSearch can be another part of the solution on the documentation/search side. This wiki is likely part of the solution. Lua may not have quite the set of libraries that Perl or Python has. However, an interesting opportunity is to expose the Python libraries into Lua via LunaticPython--instantly, Lua acquires the vast resources of the Python developer community. (Maybe the same may be done with Perl.) Making this seamless for the end-user could go a long way.