Re: embedding lua in a multithreaded application

[Hisham <h@...>]

On 18 April 2015 at 19:57, Nick Hildebrant <nick.hildebrant@gmail.com> wrote:
> Also Andres, you said one Lua thread per socket. What does that look like.
> Later you say that Lua is safe for use in multiple threads. So I'm guessing
> that one Lua thread per socket just means establishing a new state, or is
> there some other magic I'm missing?

Yes, the solution for running Lua scripts in multiple threads is to
launch multiple Lua states.

All Lua threading solutions (such as Lua Lanes) do this under the
hood, and then provide some facility to send data in between threads.
This is good for pure-Lua apps that want to use threading.

For embedding into a multithreaded app, however, usually the best
approach is to create multiple lua_State objects yourself.

-- Hisham

> On Sun, Apr 19, 2015 at 12:52 AM, Nick Hildebrant
> <nick.hildebrant@gmail.com> wrote:
>>
>> Thanks Andrew, that does help a little. I just assumed this was a common
>> problem with a common solution which might be documented somewhere. Did you
>> have any help getting you to your solution, or just read the code? So far,
>> that's all I'm really doing...
>>
>> On Sun, Apr 19, 2015 at 12:30 AM, Andrew Starks <andrew.starks@trms.com>
>> wrote:
>>>
>>>
>>>
>>> On Saturday, April 18, 2015, Nick Hildebrant <nick.hildebrant@gmail.com>
>>> wrote:
>>>>
>>>> Hi, I am a pretty new to Lua still, so please forgive me :)
>>>>
>>>> I am wanting to use lua in a multithreaded application, and I can find
>>>> lots of information about how to do threaded / psuedo threaded things in
>>>> Lua, but this is not what I am looking for. My understanding is, that the
>>>> interpreter/jit is single threaded, and I want to know how to ensure that I
>>>> am not blocking my multithreaded app by executing all my scripts in a single
>>>> thread.
>>>>
>>>> If it helps, my goal is to use LuaJIT with ZeroMQ, and have zmq manage
>>>> the parallel workflow of a bunch of services written in Lua. I have looked
>>>> at the wrapper lua-zmq, and I assume this is using lua in a truly
>>>> multithreaded way, but I don't know how to confirm. What is the trick, what
>>>> do I look for? Do we have to start an interpreter in every thread, or is it
>>>> enough just to create a new state?
>>>>
>>>> When I look at the C code where people are embedding Lua in their
>>>> programs, I don't ever really see something that looks like "the interpreter
>>>> starts running here", just setting up a new Lua state.
>>>>
>>>> Is using the LuaJIT to wrap zmq the same as embedding LuaJIT in a
>>>> program which links against ZMQ?
>>>>
>>>> I am happy to read more, I don't need anyone to write a book here, but
>>>> when I google "mutlithreaded Lua" I get a bunch of answers to a different
>>>> question (namely, "how to i make my lua script multithreaded?") which
>>>> doesn't interest me at all. I actually want to write single threaded Lua
>>>> code which will execute in the multithreaded message passing way which
>>>> ZeroMQ manages so well.
>>>>
>>>> Thanks,
>>>> Nick
>>>
>>>
>>> There are probably many patterns that you can follow here, but I use one
>>> Lua thread per socket (or group of related sockets). To communicate between
>>> Lua threads, I use nanomsg (but did use zmq, so it's basically the same
>>> scenario). I do this because I have Lua front-ending some long running
>>> processes, and so I do need the concurrency.
>>>
>>> But, you don't even need to do that, if Lua isn't the bottleneck.
>>> `select` (or better on non-windows).  You can make zmq's buffering
>>> sufficiently large, if you're trying to keep upstream processing from
>>> blocking, when Lua is busy.
>>>
>>> Does any of this provide any clues? Lua is not multithreaded but the
>>> library is careful to be safe for use from multiple threads, so long as each
>>> Lua state is being used by one thread at a time.
>>>
>>> -Andrew
>>
>>
>