Lua Token Parsing

wiki

No solutions are offered here as the whole token parsing concept is very experimental at the moment. Please create your own page, with a link from here, if you would like to offer solutions to these problems. Free free to add your own usage cases to this list.

Enumerations

Enumerations are useful for grouping values together. Enumeration can be done in Lua quite efficiently by

local SHAPE_SQUARE, SHAPE_CIRCLE, SHAPE_STAR = 0, 1, 2  -- etc.

foo(SHAPE_STAR)  -- usage

Issues:

• Enumerations are constants and we can change the values in the above example. We could use a table to hold values and use metamethods to make it read only. The overhead of this can be fairly signicant in performance (given what we are actually doing).
• Typos are not picked up until runtime despite us knowing the values at compile time.

Concept: Enum keyword and table of values which only lives at compile time.

enum ShapeType { SQUARE, CIRCLE, STAR, NUM_TYPES }

print(ShapeType.CIRCLE)  -- usage

• Enums are compile-time concept. There should be compile-time warnings if enums are not found (e.g. ShapeType.LINE).
• Enum names etc do not need to be present at runtime (at least not in the non-debug, or stripped, version) to save memory.

C Function Calls

C functions can be registered with Lua (int MyLuaFunc(lua_State*)). This registration involves passing a function pointer and a string function name to Lua (which duplicates the string). To call the function you do a global lookup using a function string name and call the object returned. More descriptive: [2].

Issues:

• If you have hundreds of functions registered this can consume a considerable amount of memory. It would be nice to remove the string registration at runtime.
• If a naming convention were introduced it would be possible to warn about function name typos at compile-time.

Concept: At Lua compile-time a list of the functions exported (much like a link library in C) could be passed to the parser. When we intercept calls to named functions we could insert the function pointer, or function enumeration in order to call the function at runtime.

Notes:

• Remove existence of (duplicate) function name strings.
• C registered functions would be const function pointers.

Switch statement

Arguably Lua doesn't need a SwitchStatement, but it might be an interesting test for ltokens. E.g. (feel free to change the syntax)

switch (value) do
    case 1 do print('one') end
    case 2 do print('two') end
    default do print('default') end
end

Try-except statement

Lua has exception support, but the interface is functional and low-level, making it cumbersome to use. Perhaps token tinkering would be helpful here.

Type checking

When calling a C binding, or other Lua function, generally objects have to be of a certain type. Lua does not support type checking, but this would be useful for these cases. For example, if we bind a C function,

void foo(int arg1, const char* arg2);

Notes:

• If an automatic binding tool is used (e.g. tolua) it could write out a metadata file containing all of the functions bound and their type information. This could be used at Lua script compilation time to check details.
• With Lua being dynamic it would be difficult to catch all function invocations, what with variables being dynamically typed. The addition of a const keyword might help here. Some variable types will be able to be predicted at compile time but others may be harder to calculate or ambiguous.

See also

• LuaMacros - a macro facility for Lua using token filters