Showing revision 195
Wiki pages
There are Lua wiki pages and projects I've been involved in.
Code and Projects:
- Formal Lua modules (core):
- Formal Lua modules (more):
- LuaToCee - convert Lua to C code using C API calls
- InlineCee - simple module that allows C code to be embedded inline in Lua code.
- LuaDist - CMake-based ports system for Lua and non-Lua packages.
- LuaFish - Lua 5.1 parser, macro system, and static type checker in LPeg [1] (related to Metalua [2])
- LuaSearch - effort to standardize Lua module documenation via POD
- ModuleReview - effort to review/classify modules in terms of quality and usefulness for Lua distributions
- SlightlyLessSimpleLuaPreprocessor - enhanced version of the enhanced version of the Lua preprocessor
- LuaProxyDll - avoid DLL dependency issue
- DetectingUndefinedVariables -
checkglobals function is an alternative to etc/strict.lua and luac -p -l - | lua test/globals.lua.
- LuaGenPlusPlus - A C++ header file to generate efficient sequences of Lua C API calls at compile-time via template metaprogramming techniques.
- LuaInterpreterInLua - reimplements the Lua 5.1 interpreter (lua.c) in Lua.
- CommandLineParsing - command line parsing (getopt)
- LuaPatch - Lua implementation of the patch utility
- ExPattern - regular-expression like support implemented in terms of Lua patterns and code generation
- SerialCommunication - serial communications in Lua/alien
- SourceOptimizer -- utility for optimizing Lua source, inlining functions
- StringQuery - string pattern matching and replacement library inspired somewhat by jQuery
- LuaAnnotate - C code comments externally stored but displayed inline.
General Topics, Patterns, Tricks, and Design:
Comments/Annotations on Other Documents:
Lua 5.2 Wishlist
These changes I want are included in the 5.2 work releases (LuaFiveTwo):
- Module system reforms - Make the module system more consistent - see LuaModuleFunctionCritiqued. 5.2 now deprecates the module function and its ilk.
- Environment reforms - I support the _ENV proposal because of the simplifications it makes to lexical analysis (LuaInspect, LuaFish/luaanalyze) and compilation (LuaToCee). I think this is related to a more general problem though involving stack levels (see level parameter below). I agree though that _ENV is ugly and probably should not be used by most user code.
- C function improvements -- Unlike lua_cpcall, avoid allocation in C functions [4] (for performance and avoiding complications in error handling) and support multiple arguments and return values in protected calls. In LuaFiveTwo, lua_cpcall is eliminated and replaced with a lua_pushcfunction that no longer allocates, and multiple arguments and return values are supported via lua_pcall.
- Lua function caching - There is now less need to manually hoist function instantiation out of inner loops since functions are now cached. However, this feature might not be implemented to its full potential[5].
- BitwiseOperators standard library. 5.2 implements this now. I had thought the Lua community had already converged on Lua Bit
Op (supported by LuaJIT and superseding lbitlib). See BitwiseOperators/[3] Both the 5.2 and LuaJIT bit libraries are probably fine for most purposes, and you could likely write code that is compatible with both: `local band = bit.band or bit32.band` provided you avoid incompatible cases. Should get/set bitfield operations be added? [6]
- __len metamethod for tables (as well as __pairs -- GeneralizedPairsAndIpairs). This causes me some hesitation though because of the questions it opens. Should we have a rawlen (similar to lua_rawlen in the work version)? Why should pairs() observe metamethods but not next()? Lua tables remain not truly virtualizable (LuaVirtualization). Should "table" functions recognize metamethods?--probably not [7].
- Expose all Lua operations to C API - Extend C API to support missing operators [8]. LuaFiveTwo now supports lua_compare and lua_arith. My main motivation for this is to simplify my LuaToCee output. Also it makes things orthogonal and is relatively easy for Lua to implement. Some things may remain lacking like tail calls in C functions [9].
- Hex escapes in strings are sometimes nice (avoiding conversion to decimal), particularly when reading files or communications streams whose specification includes bytes in hex format or whose individual bits have significance. Sure, you can use a helper function,
hex 'cdef' or binary '0101', but why need that.
- Some ask for ContinueProposal. I tend to find breaking out of nested loops more common. Here's my proposal that generalizes nested break+continue into a single construct: LuaList:2010-11/msg00556.html . See also GotoStatement
- Minor odds and ends:
- closing a pipe returns exit status
- Protect against triggering undefined behavior in C libraries from Lua standard libraries [10] like
os.date. IMO, the standard library (except maybe debug) should never crash on bad input.
- Rename lua_typerror to lua_typeerror [11]. Actually, the function is now deprecated instead.
- Other things I don't often run into but I guess are nice: more freedom to yield (including PcallAndCoroutines), xpcall accepting arguments like pcall, non-string error messages in Lua interpreter, emphemeron tables (improves weak table garbage collection).
Here's other ideas (not currently addressed in 5.2 work releases):
- `luac -l -l` is currently undocumented.
- This is a biggie. JohnBelmonte has strongly advocated it, and various people have provided patches with various implementations (above link). There's ways to do it in 5.1 with wrapper functions utilizing pcall's and anonymous functions everywhere, but it's not very nice.
- Proper error handling in the presence of resource allocation in Lua can be clumsy without this. One of the simpler solutions is a "defer" statement like in the Google Go language (or D language "scope(exit)"). Compare:
local function readfile(filename)
local fh, err = io.open(filename)
if not fh then return false, err end
local data, err = fh:read'*a'
if not data then fh:close(); return false, err end
fh:close()
return data
end
local function readfile(filename)
return scoped(function(onexit)
local fh = assert(io.open(filename)); onexit(function() fh:close() end)
return assert(fh:read'*a')
end)
end
local function readfile(filename)
local fh, err = io.open(filename); if not fh then return false, err end
defer fh:close()
local data, err = fh:read'*a'; if not data then return false, err end
return data
end
local function readfile(filename)
local fh = assert(io.open(filename)); defer fh:close()
return assert(fh:read'*a')
end
do
defer if class(err) == 'FileError' then
print(err)
err:suppress()
end
print(readfile("test.txt"))
end
local function readfile(filename)
scoped fh = assert(io.open(filename))
return assert(fh:read'*a')
end
- The
level parameter used in the error function lacks maintainability since a function might be called from different levels and those levels might be refactored. See LuaCarp for an old effort to improve this. Also see [12][13]. At least in 5.2 work, getfenv/setfenv are mostly removed and setfenv behavior in module is deprecated, so it's moving in the right direction.
function foo() error("fail", 2) end
function bar() foo() end
foo()
bar()
- StringInterpolation - There's no really good way interpolate local variables into strings. Some type of preprocessing step that expands things like
printf "${x},${y},${x}" into printf("${x},${y},${x}", x, y, x) is perhaps the ideal technical solution.
- Generalize Lua table constructor notation [14]. This also provides list comprehension syntax.
- Manipulations with
... are somewhat awkward unless you copy it in a table (not as efficient) and handle nils correctly. See also VarargTheSecondClassCitizen. Something like the proposed #... and ...[i] would help, although that particular syntax has problems. The dilemma is summarized by Roberto here: LuaList:2010-06/msg00434.html. Some small improvements are likely possible though.
- Abstracted liolib.c [15] - Many people embed Lua in systems without STDIO (e.g. GUI applications) and patch this file. Lua 5.2 provides some abstraction for this (e.g. luai_writestring) but it does not cover liolib.c.
- Maybe add versions of lua_pushstring and lua_newuserdata that don't longjump. May be useful for mixing C++ exception code and C API longjump code.
- Clearer syntax errors [16][17][18]. Some improvement is possible. Really good error/warning reporting might require an external linter.
- Add efficient tests for existence of operators on objects [19][20]
- On DetectingUndefinedVariables, my current opinion is that it is best handled by the text editor. I find runtime analysis (strict.lua) interior to static bytecode analysis (globals.lua) interior to static source analysis (LuaInspect). Furthermore globals.lua and the (hybrid) checkglobals don't work as well in 5.2 due to _ENV changes.
- Add more comments to the source [21] (LuaSource). The Lua code is small but can be difficult to penetrate. See LuaAnnotate though.
- The
assert function is somewhat misnamed: [22]. check may be better.
- Fix "package.loaders" misnomer [23][24] (package.loaders contains searchers not loaders).
- Forwardly declared local functions look instead like global definitions. Currently the best solution may be commenting:
local two
local function one() two() end
function two() one() end
- ShortAnonymousFunctions - It's often said that the Lua syntax for anonymous functions can be verbose for tiny functions used in function programming (map/filter) and functions used as blocks (e.g. simulated switch or try/catch statements). The workarounds in ShortAnonymousFunctions are not very satisfying. It doesn't seem like discussions on the user group are seriously going anywhere here though.
- Allow 'a:b()' to not depend on 'a.b' (e.g. __mcall or __mindex metamethod) [25]
- Allow pragmas in source to inject debugging info into the bytecode. [27] (useful for preprocessors of Lua source) (note: Metalua currently uses its own bytecode generator because of such limitations). There is a workaround but it is not fully general: LuaList:2010-06/msg00311.html .
-
for _,v in ipairs(t) is ugly, with the main complaint is that the meaning is not immediately apparent to users less familiar with Lua (Lua users can get used to it). The preferred syntax is for v in values(t), but the Lua for statement doesn't allow that to work with nil values.
- More efficiently (O(1)) preallocate or subsequently clear arrays. This may involve both Lua and C API functions like
lua_createtable [28]. See TablePreallocation for the preallocation (does not discuss clearing).
- The inability to incorporate statements inside expressions limits the expressibility of Lua. Well, technically you can do this via an anonymous function, but allocating anonymous functions as such is not as speed efficient (although 5.2 function caching improves it some sometimes) nor the syntax that concise (without improvements to ShortAnonymousFunctions). This can be useful in metaprogramming, and Metalua recognizes this (see "Statements in expressions" in the Metalua Manual) and extends the Lua parser/compiler to allow it while requiring no change to the Lua VM. See also the "let" statements mentioned below on this page and "Stored Expressions" in LuaDesignPatterns. The SourceOptimizer provides an automated way to inline anonymous functions into standard Lua 5.1 source code, but the transformation wasn't straightforward.
- The highly dynamic nature of Lua limits its performance. True, Lua is fast, but it could be faster. For example,
x*x is generally faster than x^2 since Lua cannot reduce the latter to the former at compile time (think: x might have a __pow metamethod). Macro processing could alleviate this (e.g. "transforming "y = SQUARE(x+y) + 1" into "y = (let t1 = x+y in t*t) + 1"). See also LuaJIT and LuaToCee. Global, or at least whole-file, flow analysis can help here.
- File handle virtualization - Sometimes you can pass a custom table that looks like a file handle to functions that expect a file handle, but this doesn't work all the time (e.g. if the function tests with io.type).
Some other things, not necessary related to Lua core development:
- Lua needs more standardization of quality control of libraries covering non-ANSI C functionality. ExtensionProposal and LuaRocks may help here. Related concerns involve documentation, testing, and packaging.
- Strength access to existing library frameworks in other platforms like .NET, Java, and Python. Lua acquires the vast resources of these other developer communities.
RecentChanges · preferences
edit · history · current revision
Edited February 23, 2012 3:46 am GMT (diff)