Wiki pages
There are Lua wiki pages and projects I've been involved in.
Code and Projects:
- LuaToCee - convert Lua to C code using C API calls
- InlineCee - simple module that allows C code to be embedded inline in Lua code.
- LuaDist - CMake-based ports system for Lua and non-Lua packages.
- LuaFish - Lua 5.1 parser, macro system, and static type checker in LPeg [1] (related to Metalua [2])
- LuaSearch - effort to standardize Lua module documenation via POD
- ModuleReview - effort to review/classify modules in terms of quality and usefulness for Lua distributions
- SlightlyLessSimpleLuaPreprocessor - enhanced version of the enhanced version of the Lua preprocessor
- LuaMatrix - pure Lua implementation of matrices
- LuaProxyDll - avoid DLL dependency issue
- LuaCom - Lua interface to Microsoft's Component Object Model (COM) - patching for 5.1
- DetectingUndefinedVariables -
checkglobals function is an alternative to etc/strict.lua and luac -p -l - | lua test/globals.lua.
- LuaGenPlusPlus - A C++ header file to generate efficient sequences of Lua C API calls at compile-time via template metaprogramming techniques.
- LuaInterpreterInLua - reimplements the Lua 5.1 interpreter (lua.c) in Lua.
- ListComprehensions - pure Lua implementation of List comprehensions via dynamic code gen
- LuaBalanced - match delimited snippets of Lua code in a string
- CommandLineParsing - command line parsing (getopt)
- LuaPatch - Lua implementation of the patch utility
- StringLibraryInLua - partially reimplements Lua's string library
- ExPattern - regular-expression like support implemented in terms of Lua patterns and code generation
- ModuleCompressDeflateLua - DEFLATE (gzip/zlib) implemented in pure Lua
- SerialCommunication - serial communications in Lua/alien
- SourceOptimizer -- utility for optimizing Lua source, inlining functions
- StringQuery - string pattern matching and replacement library inspired somewhat by jQuery
General Topics, Patterns, Tricks, and Design:
Comments/Annotations on Other Documents:
Lua 5.2 Wishlist
These changes I want are included in the 5.2 work releases (LuaFiveTwo):
- Improve lua_cpcall [7] to avoid allocation (for performance and error handling reasons) and to support multiple arguments and return values. In LuaFiveTwo, lua_cpcall is eliminated and replaced with a lua_pushcfunction that no longer allocates, and multiple arguments and return values are supported via lua_pcall.
- Protect against triggering undefined behavior in C libraries from Lua standard libraries [8] like
os.date. IMO, the standard library (except maybe debug) should never crash on bad input.
- __len metamethod for tables (GeneralizedPairsAndIpairs). However, Lua tables are still not truly virtualizable (LuaVirtualization), and there is this awkward get v.s. rawget distinction in API calls. Should "table" functions recognize metamethods? Probably not [9].
- Add BitwiseOperators to standard library. This is implemented, but I thought the Lua community had already converged on Lua Bit
Op (supported by LuaJIT and superseding lbitlib)? See BitwiseOperators/[3]
- Extend C API to support missing operators [10]. LuaFiveTwo supports lua_compare and lua_arith. There's a question on LUA_OPUNM posted to the list though. My main motivation for this is to simplify my LuaToCee output. Also it makes things orthogonal and is relatively easy for Lua to implement.
- Rename lua_typerror to lua_typeerror [11]
- hex notation in strings is sometimes nice (avoiding conversion to decimal), particularly when reading files or communications streams whose specification includes bytes in hex format or whose individual bits have significance. Sure, you can use a helper function,
hex 'cdef' or binary '0101', but why need that.
- closing a pipe returns exit status
- I support the _ENV proposal because of the simplifications it makes to lexical analysis (LuaInspect, LuaFish/luaanalyze) and compilation (LuaToCee). I think this is related to a more general problem though involving stack levels (see level parameter below).
- Other things I don't often run into but I guess are nice if I used them more: more freedom to yield (including PcallAndCoroutines), xpcall accepting arguments like pcall, non-string error messages in Lua interpreter, emphemeron tables (improves weak table garbage collection).
These features are partly included:
- Lua C API error handling and other error handling (e.g. C++ exceptions) don't play well together. I had suggested adding versions of allocation functions like lua_pushstring and lua_newuserdata that don't longjmp [12][13]. Lua 5.2 changes above concerning lua_cpcall seem to improve this some though: [4].
Here's problems and solutions not yet addressed in 5.2 work releases:
- This is a biggie. JohnBelmonte has strongly advocated it, and various people have provided patches with various implementations (above link). There's ways to do it in 5.1 with wrapper functions utilizing pcall's and anonymous functions everywhere, but I think that's a complexity most people will want to avoid.
- Proper error handling in the presence of resource allocation in Lua can be clumsy without this. One of the simpler solutions is a "defer" statement like in the Google Go language (or D language "scope(exit)"). Compare:
local function readfile(filename)
local fh, err = io.open(filename)
if not fh then return false, err end
local data, err = fh:read'*a'
if not data then fh:close(); return false, err end
fh:close()
return data
end
local function readfile(filename)
return scoped(function(onexit)
local fh = assert(io.open(filename)); onexit(function() fh:close() end)
return assert(fh:read'*a')
end)
end
local function readfile(filename)
local fh, err = io.open(filename); if not fh then return false, err end
defer fh:close()
local data, err = fh:read'*a'; if not data then return false, err end
return data
end
local function readfile(filename)
local fh = assert(io.open(filename)); defer fh:close()
return assert(fh:read'*a')
end
do
defer if class(err) == 'FileError' then
print(err)
err:suppress()
end
print(readfile("test.txt"))
end
local function readfile(filename)
scoped fh = assert(io.open(filename))
return assert(fh:read'*a')
end
- Make the module system more consistent - see LuaModuleFunctionCritiqued. Response to another proposal: LuaList:2010-06/msg00226.html . As of Lua 5.2.0work3, one part of the problem of module using setfenv was eliminated, although "To keep compatibility with old versions of Lua, module also sets t as the new environment of the current function.". This has been brought up by many people, and there is a lack of consensus on ModuleDefinition.
- The
level parameter used in the error function lacks maintainability since a function might be called from different levels and those levels might be refactored. See LuaCarp for an old effort to improve this. Also see [14][15]. At least in 5.2 work, getfenv/setfenv are mostly removed and setfenv behavior in module is deprecated, so it's moving in the right direction.
function foo() error("fail", 2) end
function bar() foo() end
foo()
bar()
- StringInterpolation - There's no really good way interpolate local variables into strings. Creating a table of locals to pass to the interpolation function is often the most practical way in Lua 5.1 but repeats oneself and involves the overhead of a table construction. Using the debug module to walk locals kind-of works but it discouraged. Some type of preprocessing step that expands things like
printf "${x},${y},${x}" into printf("${x},${y},${x}", x, y, x) is perhaps the ideal technical solution but is not built into the Lua lexer/parser and therefore currently involves patching Lua with something like token filters (though tokenf as currently implemented does not support modular programming).
- Manipulations with
... are somewhat awkward unless you copy it in a table (not as efficient) and handle nils correctly. See also VarargTheSecondClassCitizen. Something like the proposed #... and ...[i] would help, although that particular syntax has problems. The dilemma is summarized by Roberto here: LuaList:2010-06/msg00434.html. Some small improvements are likely possible though.
- Abstracted liolib.c [16] - Many people embed Lua in systems without STDIO (e.g. GUI applications).
- Clearer syntax errors [17][18][19]. Some improvement is possible. Really good error/warning reporting might require an external linter.
- Add efficient tests for existence of operators on objects [20][21]
- Improved DetectingUndefinedVariables. How does 5.2 affect this? Possible answers: [22] . I prefer the static code analysis over dynamic approaches (e.g. strict.lua). I had once proposed a mixed approach "checkglobals" (with possible with patch to
debug.getinfo or lbci). A more complete solution may be source analysis like luaanalyze (bottom of LuaFish page) and LuaInspect, which could be outside the scope of the Lua core.
- Add more comments to the source [23] (LuaSource). The small size of Lua makes it relatively easy to customize, but the lack of comments can make doing so, and doing so correctly, difficult. One needs to trace through the source to infer what functions/macros are doing and what assumptions are being made, which is time consuming.
- The
assert function is somewhat misnamed: [24]. check may be better.
- Fix "package.loaders" misnomer [25][26] (package.loaders contains searchers not loaders).
- Forwardly declared local functions look instead like global definitions. Currently the best solution may be commenting:
local two
local function one() two() end
function two() one() end
- ShortAnonymousFunctions - It's often said that the Lua syntax for anonymous functions can be verbose for tiny functions used in function programming (map/filter) and functions used as blocks (e.g. simulated switch or try/catch statements). The workarounds in ShortAnonymousFunctions are not very satisfying.
- Allow 'a:b()' to not depend on 'a.b' (e.g. __mcall or __mindex metamethod) [27]
- Allow pragmas in source to inject debugging info into the bytecode. [29] (useful for preprocessors of Lua source) (note: Metalua currently uses its own bytecode generator because of such limitations). There is a workaround but it is not fully general: LuaList:2010-06/msg00311.html .
-
for _,v in ipairs(t) is ugly, with the main complaint is that the meaning is not immediately apparent to users less familiar with Lua (Lua users can get used to it). The preferred syntax is for v in values(t), but the Lua for statement doesn't allow that to work with nil values.
- More efficiently (O(1)) preallocate or subsequently clear arrays. This may involve both Lua and C API functions like
lua_createtable [30]. See TablePreallocation for the preallocation (does not discuss clearing).
- The inability to incorporate statements inside expressions limits the expressibility of Lua. Well, technically you can do this via an anonymous function, but allocating anonymous functions as such is not as speed efficient nor the syntax that concise (without improvements to ShortAnonymousFunctions). This can be useful in metaprogramming, and Metalua recognizes this (see "Statements in expressions" in the Metalua Manual) and extends the Lua parser/compiler to allow it while requiring no change to the Lua VM. See also the "let" statements mentioned below on this page and "Stored Expressions" in LuaDesignPatterns. The SourceOptimizer provides an automated way to inline anonymous functions into standard Lua 5.1 source code, but the transformation wasn't straightforward.
- The highly dynamic nature of Lua limits its performance. True, Lua is fast, but it could be faster. For example,
x*x is generally faster than x^2 since Lua cannot reduce the latter to the former at compile time (think: x might have a __pow metamethod). Macro processing could alleviate this (e.g. "transforming "y = SQUARE(x+y) + 1" into "y = (let t1 = x+y in t*t) + 1"). See also LuaJIT and LuaToCee. Global, or at least whole-file, flow analysis can help here.
- A Lua table cannot impersonate a file handle (or stream). For example, you may have a function that writes to a given file object. You might want to pass to that function not a real file object but an object with the same interface as a file object and having an implementation that does something else (e.g. appends to a string buffer). One use of this is would be to redirect standard output not to a file but rather to a text box in a GUI.
Some other things, not necessary related to Lua core development:
- Lua is in need of optional interfaces to non-ANSI C functionality (e.g. file directory listing) that are more accessible. ExtensionProposal and LuaRocks may help here. Real programs often need non-ANSI functionality, so why not standardize that functionality to the extent that it can be standardized?
- There is much need for a more standard way to document Lua modules. Perl has POD, and that was proposed in Lua in LuaSearch, but it hasn't yet catched on. Natural Docs has some benefits but requires some patches and has some questions of its own [5]. The benefits of these two documentation formats is that they are language neutral. (I have more to say on this.) Some want Python-like docstrings queryable at runtime though (see "Decorators, Annotators and Docstrings" in LuaDesignPatterns).
- There is a lack of consistency and centralization for Lua modules--in terms of locating, installing, documenting, testing, dependency tracking, etc. Think: Perl's CPAN. Others have noted this too [6]. [LuaForge] is part of the solution. LuaRocks and LuaDist may be part of the solution too. I believe that LuaSearch can be another part of the solution on the documentation/search side. This wiki is likely part of the solution. Lua may not have quite the set of libraries that Perl or Python has. However, an interesting opportunity is to expose the Python libraries into Lua via LunaticPython--instantly, Lua acquires the vast resources of the Python developer community (the same may be done with Perl, .NET and Java). Making this seamless for the end-user could go a long way.
FindPage · RecentChanges · preferences
edit · history
Last edited July 24, 2010 11:32 pm GMT (diff)