Lua Library Workshop

lua-users home

Lua Library Design Workshop at Harvard, Tuesday, February 19

The design of the embedded programming language Lua [1] shows the results of almost ten years' worth of careful thought and attention from the designers. The standard library that accompanies Lua, however, has not received comparable attention. As users of the language, we have an interest in improving the library. We are therefore holding a (short) one-day workshop to develop suggestions for improvements in the library. Backwards compatibility is not a requirement; we are free to scrap everything and start over if we want! (We will stick to the current language definition, however---only the library is up for grabs.) We will be fortunate enough to have with us RobertoIerusalimschy, one of Lua's chief designers.

We anticipate working from about 10 to 4, with a short break for lunch. This is intended to be a working meeting, not a day of talks and panels, and we hope to emerge with very concrete, specific suggestions for the Lua team. For that reason, enrollment will be limited to about 20 people.

Please let us know if you would like to participate in this workshop. Send email to Christian Lindig []. We would like to hear from you by Friday, February 8.

-- ChristianLindig?

Please suggest what you would like to see discussed at the workshop, even if you cannot attend.

ReubenThomas continues: Before starting it is important to establish the scope and purpose of the standard libraries. In my opinion, the standard libraries should

ChristianLinding? originally listed some recent topics from the mailing list below, most of which are probably not suitable for discussion at the workshop; here's the list, with explanation:

RiciLake interjects:

I agree with ReubenThomas that the standard libraries should only provide services which could be written in portable ANSI C. However, it also seems clear that many of the enhancements suggested by ChristianLindig? would be useful to many people, and indeed have been requested many times. There is perhaps a middle ground here: define the standard interfaces to a set of facilities commonly available on many operating systems and possibly provide a reference implementation for one of them. A well-thought-out set of interfaces is often the hardest part of library design.

This would at least allow people to write semi-portable code; that is, code portable to platforms for which the standard interfaces have been implemented. Candidates for such interfaces would probably include IP sockets, file attributes, a subset of the rest of POSIX, and dynamic linking facilities, to select a few things from Christian's list.

Ports of the standard interfaces would be highly encouraged, but with some moral suasion against over- or under- implementing them. The spirit would be "if you have the facility, this is how you integrate it into Lua." Then a simple existence test for the implementing function would suffice to know whether the facility was available; it would be nice if it were never necessary to perform tests by operating system. This is somewhat limiting but not as limiting as pure ANSI C (and, in fact, compatible with the current philosophy where the math or io libraries may not be available for particular embedded platforms.)

The one thing I forgot from when I wrote this the first time: it would be really nice if the math library made it possible to use IEEE-754 arithmetic on platforms which support it. It wouldn't break my heart if traps were not supported, but it would be nice to be able to test and set the IEEE-754 mandated exception flags, and to have a way of at least turning traps off if they cannot otherwise be supported. And of course to test for NaN and +-Inf.

Back to ReubenThomas...

Here are some more high-level ideas of mine:

1. Is it possible to expose more of Lua's internals as libraries?

In particular, it'd be really nice to be able to get at the internals of the compilation process, e.g. to be able to manipulate code as data (rather than resorting to string mangling as at present). Similarly, see my past proposals on the list about being able to plug in a different regexp engine or (more controversially) garbage collector.

2. What about reimplementing some of the standard libraries in Lua?

Some functions, like assert, could easily be implemented in Lua, and implementing them in C gives little or no advantage. In addition, there are functions written in Lua that it is most useful to add pre-loaded to the interpreter, e.g. require (to 4.0). Having a framework to build Lua libraries into the executable would be easy (bin2c already does the work) and useful. Reducing the amount of code written in C would improve usability and reduce maintenance.

3. What else is needed?

As far as the standard libraries go, this reduces to the small question of whether there are any missing bits of the ANSI C libraries that can usefully be exposed to Lua, and the rather bigger question of whether there are any techniques that, like pattern matching, would both benefit from a C implementation and be widely used. The only (pure ANSI) functionality I've ever needed to add has been bitwise logical operations (my library for this is widely packaged, so others seem to agree).

Finally, an obvious next step if you accept the argument that as little as possible should be written in C is:

4. What about Lua standard libraries written in Lua?

I'm not talking here about a large rich set of libraries as in the StandardLibraryProposal, but just having some core functionality built into Lua. Candidates for this include functions such as assert and require which neither functionality nor performance reasons force to be implemented in C.

One specific idea: it would be useful to have a fuller interface to strftime so that one can, for instance, use it to construct lists of the names of the months or days in the current locale. Being able to pass nil (NULL) to setlocale would also be useful.

List Implementation (NickTrout)

The implementation of lists using Luas tables ( tinsert(), tremove() and getn() ) involves adding "n" to each table to hold the highest index of elements included. This addition has a few problems :-

list = { "a","b","c" }

for _,v in list do print(v) end -- will print a,b,c,d,4
foreach(list, function(i,v) print(v) end) -- will print a,b,c,d,4
for i = 1,getn(list) do local v=list[i] print(v) end -- will print a,b,c,d
foreachi(list, function(i,v) print(v) end) -- will print a,b,c,d
The first method for iterating over a list is clear and convenient but fails due to n. The last two methods do work, but I don't think they are as concise (or efficient?).

Now that unified method tables have been added wouldn't it be good to deprecate tinsert, tremove and getn and add a table() method which creates a table list with functionality similar to PythonLists and PythonDictionaries combined. It would probably be called "table" as opposed to "list" or "dictionary" as the functionality is a union of the two. eg.

t = {}  -- table with no methods, "clean"
t = table()  -- create table with tag/unified methods attached.
t = table{ a=1,b=7,c=8 }  -- constructor with dictionary/associative items.
t = table{ a=1 ; "b","c" }  -- mixed items.
t = table{ "a","b","c" }  -- constructor with list.

print( t:len() )
t= t:sort( function (a,b) return a>b end )
print( t:count("b") )
t:print()  -- print items (rather than "<table>") and/or "print( t:tostring() )"
for _,v in t do print(v) end  -- and no "n"!

Some thoughts of PeterPrade about the standardlibrary:

Instead of going this way, I think it's better to just accept Lua's lists as "raw lists". These raw lists are used by the VM for function arguments, and of course are the result of table constructors. The implementation is exposed (the n field). Many applications will be better served by a higher level list implementation, as you propose. But certainly you'd want the higher level list implemented on top of whatever class system you were using. I don't think the Lua distribution should be imposing a class system. So this kind of thing belongs in a bigger "on top of Lua" library... --JohnBelmonte

Some sort of basic table functionality, n, will need to stay, so that for i=1,getn(t) ... end will work for the most basic scripts. In any event, adding n to the table is not popular and should be removed. Whether the above table functionality comes as an optional component of the standard library or part of an extended library is the topic of discussion. I can see your point about it imposing slightly, but since it is very basic I cant really see it imposing too much to the extent that its the basis of a class system. --NickTrout

Summary From the Workshop

The workshop started with a public talk by Roberto Ierusalimschy on Lua. He stressed the design philosophy of Lua. The talk also made obvious that Lua's small size and fast execution is a result of a well thought out implementation.

After the talk the following people attended the Lua Library Workshop:

Here is an incomplete list of topics that were discussed at the Lua Library Workshop. Maybe the other attendees of the workshop can help to complete it.

-- ChristianLindig?

RecentChanges · preferences
edit · history
Last edited July 10, 2006 5:55 pm GMT (diff)