Re: Multithreading and locking...

[Francisco Olarte <folarte@...>]

On Tue, May 21, 2019 at 12:07 PM Victor Bombi <sonoro@telefonica.net> wrote:
> Personally I prefer using lanes for running a different Lua_State in every OS thread (called a lane) and getting comunication between lanes with the lanes API.

My main problem is I do not have control of who calls the interpreter,
in heavily depends on runtime state. The app uses thread pools to do
it's work and sometimes calls back to my core logic in the
interpreter.

I've read of lanes before, reread it now and saw the first paragraph,
"Lua Lanes is a Lua extension library providing the possibility to run
multiple Lua states in parallel. It is intended to be used for
optimizing performance on multicore CPU's and to study ways to make
Lua programs naturally parallel to begin with. " I do not need, and
explicitly do not want, multiple states, and have no performance
problem, I'm going to use less than 1% of a single core in lua. Lua
code is going to control C code which goes in it's own thread pool,
just making the interesting high level executive decisions, not doing
the low level stuff.

> You say that
> > For a program I'm doing I need to use a single interpreter from
> > multiple threads.
> Are you sure you need a single interpreter?

I do not strictly need it, I can just code the core logic in C++ and
use zero interpreters, and reload dso for reconfiguration, or use a
shared data structure and shuttle data around and store interpreters
in thread locals, or use a lua interpreter for each thing. But the
problem I try to solve greatly benefits from a shared interpreter. The
lua code becomes easier. It normally just turns events into coroutine
dispatchs for handling some things, like IVR call flows, and
manipulates some global state for others, like a routing decission.
Having all this in a single interpreter means nothing moves while we
are inside lua, and the lack of syncing while inside means I get in &
out very fast, simple code, small bug surface.

But in some corner cases I MAY have to reenter. The typical case is
what I more or less told, I call a C function which sends a message
which ends up in another thread which wants to, say, read some data (
calling a func ) in the interpreter. Concurrency is not a problem. I
would like to do that by temporarily unlocking, but I think I can
manage it which yields, after all I'm going to have a dedicated C++
module for it, and encapsulate potentially reentering functions into
some objects having a before-unlock, unlocked, after-unlock virtual
methods. I want to just call them, but I think with very little
overhead I can code the interpreter entering code as a small loop
which handles LUA_YIELD and dispatch it using lua_yield() from some
stub transparently in the C++ side.

Francisco Olarte.