Hello all,
We've found ourselves unexpectedly using Lua as the logic engine for many of our worker threads and replacing business logic that would otherwise be modeled in C++. In this role, Lua exists in a concurrent world and it does this very well, as long as the limitations are understood.
These limitations promote certain patterns, such as message passing. For example, you must respect Lua's single-threaded nature. Most people do this by using (at least) one Lua state per OS thread. This leads to passing by value and not by reference and so a message passing pattern emerges.
When we set out to connect Lua states together, we decided to use nanomsg for passing the messages. For actually making the messages themselves, we "punted" and simply used Penlight's "pretty.write" function (plus a little extra code thrown in for convenience). This has been fine during testing, but now we need to pass userdata and to carry along structured information more naturally (metatables, uservalues, etc).
Out of ignorance, perceived necessity or actual necessity, we've been doing this by hand. There are some awesome projects out there that have already implemented most of what we need, but they feel a bit "high-level", in that they come with an architecture if their own.
Generally, serialization of data between lua_states is simple enough, but I have a suspicion that there could be a few things that could be made simpler. For example, there is xmove, but it doesn't work between states. While the function call will go without a hitch, string interning will mess up equality. This isn't a big deal, because translating strings is fast and simple, but it does force you to translate a table by "hand" or within your own function.
Things are trickier with functions and upvalues. On first blush, it seems like it's a matter of collecting everything that is needed. But even I can see that this would lead to enormous bloat, without some kind of caching mechanism. All of that leads me to wonder about the need for constants. Then I wonder if transferring functions is a good idea at all...
Some questions:
Are there some simple facilities that facilitate some of the more mundane tasks associated with transferring values and variables between Lua states? Specifically: userdata, tables and the associated structure of their metatables and uservalues.
Are there ideas from Clojure, Erlang or Go that could be lightly borrowed?
Is there any bookkeeping code that you find yourself writing that would be better of done in a general library?
-Andrew
