lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

Regarding serialization, I think latest Lua Lanes did away with most of it, pushing/popping data directly between the parent and the child states.

The Lanes approach is one of OS thread per each Lua state, but I tried to get the most of it. Unfortunately, my late web page cleanup seems to have torn the online manual away. It is within the tgz file available in Luaforge, though.

May or may not be helpful to you.


On Tue, 1 Apr 2008 02:08:09 +0400
 "Alexander Gladysh" <> wrote:
Hi, list!

I'm writing a generic task server. Each task is a Lua coroutine. Server runs several system threads, each thread with separate Lua state. Each state may contain many tasks. New tasks are added to the least loaded thread. Preliminary analysis suggest that performance would benefit from some more fine-grained load balancing on the server -- make each task thread-independent, to allow next available thread
to update next available task.

This is not easy thing to do though.

Simplest solution would be to keep a separate Lua state for each task (possibly with cache of states). This would require almost no modifications to existing code. However, I'm afraid (haven't tested yet) that this solution is too heavy as I have too many script files to read, compile and execute on state creation (okay, no compile step with precompiled code; also can keep all sources cached in memory... but still feels like too much). I thought about loading single Lua state and then cloning it with lper. However this is Linux-only solution. Furthermore, LPSM seems to be abandoned long ago. Have any
one used it in production environment?

Then there is an option to keep single state per thread and to copy task data between them. This looks even less viable. My tasks are sometimes data-heavy (up to few megabytes), and I expect serialization and deserialization on each tick to take too much time. Also, this imposes restrictions on what may be serialized -- I'd like to avoid using low-level patches/libraries like Pluto. What-to-serialize problems are not too much complex to workaround, but would require some painful code changes -- again, hard to expect serialization
performance to be good enough.

I have never looked close enough, but looks like existing solutions like Copas and Helper Threads Toolkit are not exactly fit to my problem. Perhaps there is something else that I've missed? Any recent
advances in the field? Any advice?

Perhaps there is some way to implement some performance-effective cross-state Lua table? Looks like such thing (albeit filled with native Lua data only, and no coroutines/functions with upvalues etc.) would benefit implementing second solution by eliminating the need of serialization. But then, I guess, even *if* it would be possible, it
would involve some nasty core source patching. :-)