lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]


On 01/27/2014 03:45 PM, Andrew Starks wrote:


On Monday, January 27, 2014, Dirk Zoller <duz@sol-3.de
<mailto:duz@sol-3.de>> wrote:

    Hi all,

    in my application, many many little problems are tackled by multiple
    threads. Currently, I have a lua_State per problem domain and
    thread. This is in the order of thousands of lua_States which work
    in like 16 threads on those many little problems.

    I could do with only one lua_State per thread.
    That would make my program less complex which is better.

    Before I change my program towards fewer lua_States, I'd like to
    hear your opinion on this. Will the reduction of thousands of
    lua_States, each working with one script on a small problem, down to
    one lua_State per thread, each working with several scripts on many
    small problems, will this be beneficial or harmful?

    Does Lua suffer from scalability problems which I didn't run into
    yet because I chopped the work into so many small pieces?

    Thank you very much for any insights you can share on this.

    Dirk


I'd be interested in hearing this as well. Also, I'm interested in your
findings. Thousands? That's many more than I thought reasonable. What
kind of hardware are you running? Are the threads often idle? How are
you tackling communication?

I've been toying with ways to make a nanomsg socket interface  look like
and work like a thin wrapper around coroutines. That's as much insight
as I have to offer, unfortunately.

-Andrew

Think of a spreadsheet where in some cells the user can run a script to process data. A lua_State is created per line. People build larger setups than the developer anticipated. Typically, each script terminates within micros. Yet there are many and they are called frequently.

It runs on a 16 cpu server. It runs well, load is nicely balanced. Problems are spurious lockups or crashes. I want to simplify it. It has grown to complex.

Now if I redesign it to use fewer lua_States that would be nicer. But I'd rather not then see performance hickups come in and learn that the original design was actually better.