Lua Power Patches

lua-users home
wiki

A power patch is a small patch to a source code distribution that makes some useful change. Power patches are judged based on how few lines of code are changed, general usefulness, and backwards compatibility. By limiting them to localized changes, the chance that several such patches can work together is high. It also keeps maintenance work to a minimum as new versions of the source package are released. Truly good patches will have a short life, as the authors of the original program will incorporate them into their code.

New power patches, ports of existing patches to different Lua versions, and bug fixes are welcome.

If you apply a patch that changes the syntax or semantics of Lua, the resulting language should not be called "Lua".

See LuaPowerPatchesArchive for patches for old versions of Lua. Where a patch exists for multiple versions of Lua including the current one, it will appear on this page.

How to Apply a Patch

Unpack the correct Lua distribution into a clean directory. Do a cd to the top of the directory and run:

patch -p1 < patchfile

Patch Tools

Patch Guidelines

cd lua
make clean
cd ../lua_new
make clean
cd ..
diff -urN lua lua_new > mychange.patch


Available Patches

These are roughly in order of Lua version, newest to oldest.

"checkglobals": Check for Undefined Globals

This patch is described in DetectingUndefinedVariables.

Continue Statement for Lua 5.1.3

A "continue" statement is added to the parser. The virtual machine is unchanged. Revised to the current Lua version with a small test suite.

Instead of patching Lua, one might consider luaSub, which contains a syntax mod for this same purpose.

Table Scope patch

Allows table constructs to enclose a scope so that variables used inside the table have special meaning. See TableScope.

LNUM - number mode patch ("integer patch"), revised to Lua 5.1.3

Allows Lua built-in numbers to be a combination of any of the following: LNUM_DOUBLE / LNUM_FLOAT / LNUM_LDOUBLE (long double) LNUM_INT32 / LNUM_INT64 LNUM_COMPLEX

AKa 19-Mar-08: Revised for Lua 5.1.3, see LuaForge. AKa 3-Oct-07: Revised, see [News]

Uses: 32- or 64-bit integer accuracy internally for any Lua numbers. Intensively (40-500%) boosts Lua performance on non-FPU platforms. Totally transparent to the application (script) level, as well as existing Lua/C API. Even the use of complex numbers is.

Latest svn (with test suite):

svn export svn://slugak.dyndns.org/public/2008/LuaPatches/LNUM2

Latest release: [LuaForge LNUM Files]

Real-world testing and performance data from true applications are still appreciated. The patch is essentially "ready"; apart from LDOUBLE mode there are no known bugs; if you find any, please share the info.

The patch leaves integer realm graciously, falling into floating point accuracy if results won't fit in integers. The revise of 2008 boost speed further by no longer reducing speed of floating point calculations.

For performance results, there is a spreadsheet and easy to use "make-plain/float/double/ldouble/complex" targets to run on your own system.

Integer ascii values

Yet another syntactic bloat for the lexer:

print(#'a',#'\n')
97      10

note that this could be probably done via token filter as well, but .. useful for various ascii value mangling (base64,obscure protocol parsing..) .. i just wanted something more expressive than if c>=b2a("A") and c<=b2a("Z"). there was no way to get ascii value as a constant in lua to this point.

This syntax already has a meaning (string length) because strings can be given inside single quotes. --lhf

Yieldable For Loop

Modifies the code generator so that the iterator in for ... in loops can call yield. Details and test code are available at YieldableForLoops.

Note that the current version of the patch orders op codes so as to maintain binary compatibility with compiled Lua scripts, if LUA_COMPAT_TFORLOOP is defined in luaconf.h. This adds a few instructions to the VM, and quite a bit of complexity to the patch, which would otherwise only be about 25 lines modified.

Module Execution Proposal

Provides a new command-line switch (-p) that loads a function with the given package name via the searchers in package.loaded and then executes that function as a script, passing the command-line arguments to the function as arguments. Patch and description are in ModuleExecutionProposal.

Better signal handling in the interpreter on POSIX systems

Use sigaction instead of signal. This means that, for example, you don't have to press Ctrl-C twice to quit a Lua script blocked on I/O.

Remove auto string<->number conversion

Prevent auto-conversion between strings and numbers in arithmetic and concatenation. This is good because it prevents bugs; when auto-conversion is a good idea (as in the print function) it can still be done by calling the relevant conversion functions.

The current version does not scrupulously remove all undesirable casting from the libraries; I have preferred correctness over completeness. I now have a plan for this, and have started looking through the sources.

Print NULs

Make print print NUL characters, by using fwrite instead of fputs. Someone else tidied it up by patching luaconf.h to let the user supply a luai_puts macro.

Save readline history

From the standalone interpreter, save the readline history in a configurable file (defaults to .lua_history on POSIX, lua_history.txt on Windows).

Use defaults for LUA_INIT, LUA_PATH and LUA_CPATH

Adds a -t switch to the standalone interpreter that uses the default values for the above variables, making it easier to run Lua in a controlled way.

Concise anonymous functions

Some syntactic sugar for writing more concise anonymous functions, very useful when passing arguments to higher-order functions. Completely backwards compatible and only modifies the parser, emits the same bytecode as the old syntax. See [The Readme] for details.

Extend table constructor syntax to allow multiple expansion of multireturn functions

As discussed several times on the mailing list, and implemented in Aranha, here is a patch which modifies table constructor syntax so that ; allows the preceding list item to be expanded.

With this patch, {foo(); bar()} creates a table with all the return values of foo followed by all the return values of bar, while {foo(), bar()} continues to have the same semantics as current Lua. To be more precise, if a list-item is followed by a comma, then it is truncated to one return value; otherwise it represents all the return values (possibly none). Consequently, {foo(),} truncates, but {foo();} and {foo()} do not.

The patch also makes the order of field definitions precisely left to right, unlike the current Lua implementation in which {[3] = "foo", 1, 2, 3} has undefined behaviour. That may lead to performance problems with very large tables which are defined like this:

t = {
  "a", a = 1,
  "b", b = 2,
  "c", c = 3,
  -- etc
}

Other than that, the performance implications are minimal; sometimes it is a bit faster, sometimes it is a bit slower, but there is little difference. Personally, I prefer the precise ordering guarantee, but the patch can be easily modified to come closer to current semantics. Contact me for more details.

The implementation is straight-forward. When a semi-colon is encountered, the compiler emits code to append the current list of expressions, leaving the last one as multi-return. In order to do this, it's necessary to keep the current table array insertion point on the stack, instead of hard-coding it into the vm code, so the opcode to create a new table is modified to use two stack slots, putting the table in the first one and initializing the second one to 1. The opcode which adds array values to a table uses the second stack slot as the starting index, and updates it to the next starting index. Although this uses one extra stack slot, it is roughly the same speed as the existing code.

Let files combined with luac access arguments

When luac combines multiple files into a single bytecode chunk, the resulting chunk does not accept any arguments. This small patch passes ... into all files in the combined chunk

Autotoolized Lua

This patch autotoolizes Lua distribution, i.e. makes it use autoconf, automake, and libtool. For Lua 5.1 and 5.1.1.

You must unpack the file before patching:

bunzip2 lua-X.Y.Z-autotoolize-rW.patch.bz2

Next, apply the patch.

After the patching you need to add executable flag to some files:

chmod u+x autogen.sh config.guess config.sub configure depcomp install-sh missing

Now you are ready to run ./configure.

__usedindex metamethod

__usedindex behaves exactly like __newindex but when the indexed key actually exists (value is overwritten). this allows simple implementation of read-only tables, mirroring C structures etc without slow/lengthy/fragile table proxying constructs. Known to be broken with LuaJIT, fixes are welcome.

experimental php-like 'break N' to break across multiple loops

allows syntax like: while foo do while bar do if baz then eek() break 2 end end end

if "foo and bar and baz" condition holds, 'break 2' escapes loops imediately. the number is counted towards breakable scopes (thus "if", "do" etc are left out)

added bitwise operators, integer division and != for Lua 5.1.1

All these features can be disabled with undefining LUA_BITWISE_OPERATORS in luaconf.h.

Bitwise operators first convert a lua_Number to a lua_Integer and convert back the result to a lua_Number.

string.format %s patched to use __tostring (for Lua 5.1.1)

Changes string.format %s to apply __tostring to non-string %s arguments.

Enum/bit operations patch (to Lua 5.1.1)

Adds C API functions (toenum, isenum, ..) for handling unsigned 32-bit bitfields. Such enum values have overloaded [], () operations for performing bitwise operations on the Lua side. Enums are grouped in 'families', to prevent accidential use of wrong bitmask in wrong function.

The implementation uses negative 'tt' (Lua type) values for enums, and they are not garbage collectable (= should be fast). 'type()' function returns two values: "enum" and the family name. Documentation is lacking.

svn cat svn://slugak.dyndns.org/public/lua-bitwise/lua-5.1.1-enum-patch.diff
svn cat svn://slugak.dyndns.org/public/lua-bitwise/test.lua
svn cat svn://slugak.dyndns.org/public/lua-bitwise/README

Go Long Lua!

This patch removes floating point operations used by Lua 5.1 by changing the type of Lua numbers from double to long. It implements division and modulus so that x == (x / y) * y + x % y. The exponentiation function returns zero for negative exponents. The patch removes the difftime function, and the math module should not be used. The string.format function no longer handles the floating point directives %e, %E, %f, %g, and %G. By removing the definition of LUA_NUMBER_INTEGRAL in src/luaconf.h, one obtains a Lua number implementation based on doubles.

Unpack Tables by Name

Enhancement to the assignment statement to unpack named values from tables using the in keyword. (See lua-l message "patch: local a,b from t" [4].)

    local a, b, c in some_table_expression
is syntactic sugar for
    local t = some_table_expression
    local a, b, c = t.a, t.b, t.c

Custom error object support

This patch improves Lua's support for custom error objects. Changes:

See "Exception Patterns in Lua" [3] for more information.

CNUMBER patch, Lua 5.1

Provides a more efficient mechanism for accessing numeric C variables from Lua.

Do patch, revised to Lua 5.1 (beta)

Makes "= do ... end" be syntactic sugar for "= function() ... end" (handy with simple callbacks).

svn cat svn://slugak.dyndns.org/public/lua-patches/do.patch
svn cat svn://slugak.dyndns.org/public/lua-patches/do.txt

Instead of patching Lua, one might consider luaSub, which contains a syntax mod for this same purpose.

Literals (hex, UTF-8) patch, revised to Lua 5.1 (beta)

Allows \x00..\xFFFF (hex) and \u0000..\uFFFF (UTF-8 encoded) characters within strings.

svn cat svn://slugak.dyndns.org/public/lua-patches/literals.patch
svn cat svn://slugak.dyndns.org/public/lua-patches/literals.txt

Equality operators that work like arithmetic operators

This modifies the behavior of the equality operator functions so they are able to handle values with dissimilar types. For instance, in standard Lua if the left operand is a userdata and the right is a number, the equality test will fail. This patch causes the __eq metamethod of the userdata to be used, if available. But note, one reason Lua does not support this is because the __eq, __lt and __le metamethods are used for ~=, > and >= as well, by reversing the operands. Therefore, if both the right and left operands have metamethods, you might be surprised by which one gets chosen. As it is, the left metamethod is preferred. But of course, this is the RIGHT metamethod for the ~=, > and >= tests! A good solution to this might be to add __ne, __gt and __ge metamethods. Then the equality operators would truly behave exactly like the arithmetic operators.

Mutate Operators

This patch adds mutate operators to Lua. Specifically, the ":=" operator can now be used for value assignment (or whatever else you wish) by attaching a "__mutate_asn" metamethod to a Lua object. Adding additional mutate operators (such as +=, or << for example) is straightforward.

Advanced readline support

This patch adds the following features to the existing readline support in Lua 5.x:

After applying the patch start lua and try these (replace ~ with the TAB key):

~~
fu~foo() ret~fa~end<CR>
io~~~s~~~o~~~w~"foo\n")<CR>

It has been verified to work with Lua 5.0, 5.0.2, 5.1 and GNU readline 2.2.1, 4.0, 4.3, 5.0, 5.1 or NetBSD libedit 2.6.5, 2.6.9.

Bitwise operators and hexadecimal support

This patch adds the following features to Lua 5.x:

NOTE: This patch adds opcodes to the Lua virtual machine. The two major consequences of this are that (1) the result cannot be called 'Lua', and (2) compiled code will not run on interpreters that do not include this patch.

After applying the patch, try running the following:

> hex=function(n) print("0x"..string.format("%X",n)) end
> hex(0x54&0x55)
0x54
> hex(0x54|0x66)
0x76
> hex(0x54#0x66)
0x32
> hex(#0x45)    
0xFFFFFFBA
> print("Hel\x6c\x6f world\x21")
Hello world!
> 

This has been tested on Lua-5.0.X; separate patch for 5.1-work0

BUG-REPORT/NOTE: This patch uses the C function strtol(), which on most systems has a maximum input value of 0x7FFFFFFF, so the command hex(0x97F2DA31) will output 0x7FFFFFFF. [ This is easy enough to fix on machines that support 64-bit integers (i.e. long long) by using strtoll() (Posix) or __strtoi64() (Win32) ].

Kilobyte/Megabyte Number Suffix

Add 'K' and 'M' suffixes for numbers (e.g. 150K or 12M); binary (K=2^10) not metric (K=10^3).

Useful when using Lua as a configuration language in some domains (in our case, a build process).


FindPage · RecentChanges · preferences
edit · history
Last edited March 23, 2008 3:53 am GMT (diff)