Re: Scalability of Lua, many small vs. few big lua_States?

[Andrew Starks <andrew.starks@...>]

On Monday, January 27, 2014, Dirk Zoller <duz@sol-3.de> wrote:
> Hi all,
>
> in my application, many many little problems are tackled by multiple threads. Currently, I have a lua_State per problem domain and thread. This is in the order of thousands of lua_States which work in like 16 threads on those many little problems.
>
> I could do with only one lua_State per thread.
> That would make my program less complex which is better.
>
> Before I change my program towards fewer lua_States, I'd like to hear your opinion on this. Will the reduction of thousands of lua_States, each working with one script on a small problem, down to one lua_State per thread, each working with several scripts on many small problems, will this be beneficial or harmful?
>
> Does Lua suffer from scalability problems which I didn't run into yet because I chopped the work into so many small pieces?
>
> Thank you very much for any insights you can share on this.
>
> Dirk

I'd be interested in hearing this as well. Also, I'm interested in your findings. Thousands? That's many more than I thought reasonable. What kind of hardware are you running? Are the threads often idle? How are you tackling communication? 

I've been toying with ways to make a nanomsg socket interface  look like and work like a thin wrapper around coroutines. That's as much insight as I have to offer, unfortunately. 

-Andrew