RE: Simple Function Call From C language

[Thijs Schreijer <thijs@...>]

> -----Original Message-----
> From: lua-l-bounces@lists.lua.org [mailto:lua-l-bounces@lists.lua.org] On
> Behalf Of Austin Einter
> Sent: dinsdag 24 juni 2014 14:54
> To: Lua mailing list
> Subject: Re: Simple Function Call From C language
> 
> Hi Thijs Schreijer
> 
> You told
> > You can only acccess a lua state from a single thread. So assuming you
> > stay single threaded, then for each invocation you can simply clear the
> > stack, and do your thing. Nobody will interfere with that.
> 
> In above are you telling that lua state can be accessed from a single
> thread. If in one process I have 10 threads, and from 10 threads I am trying
> to access lua state, will it be a sequential execution hence a performance
> issue. will not 10 threads access lua state in parallel.

Technically speaking; it only would if you run on a multi-core cpu machine. But considering the numbers you mentioned, you probably are.

> 
> In another case when I have 1000s of processes, and say that from each
> process a single thread is accessing lua state. Is it that only one process
> can access lua state at a time, again a risk of sequential execution of
> processes.
> Kindly clarify on above.

Have a look here regarding states and threads; http://www.thijsschreijer.nl/blog/?p=693

> 
> 
> On sandboxing question, I hope you preferred "do luaL_openlibs once at C
> code init time and lua_close
>   at C code exit time" .  Though it looks it will be preferred one from
> performance perspective, what are other implications. Since I have 1000s of
> processes, how much system resources like semaphore, shared memory, mutex
> etc will be present per lua state. What extra RAM usage will be there per
> lua state keeping it active throughout.
> When we have 1000s or 10,000s processes running in a machine, can I really
> dare to go for above approach.

First of all; though I never implemented something like this myself, from following this mailing list for a few years now I know that this is NOT an uncommon scenario, many have implemented similar things. I don't recall who posted it (nor could find the related message/thread) but a while ago someone wrote how he implemented a large scale parallel processing similar to your case.

He setup a pool of Lua states, limited by two constants, for minimum and maximum number. Each thread requiring Lua processing would get a state from the list (if not available one would be created and initialized with libraries etc., until the maximum set was reached). When processing finished, the state was released back into the pool of states, ready to be used by another thread.
Count of usage was kept and once every few seconds a check was done to ensure that if states had been unused for a while they would be destroyed (until the minimum set was hit)
This scenario reuses the same states instead of creating and destroying new ones all the time.

Lua itself doesn't use mutexes or semaphores, so in the described scenario you would only need to protect the pool of lua states with a mutex (or maybe even a lockless list).

Regarding the RAM usage; the Lua engine stores all data in lua states it creates. I don't know the size of an initialized state, but that should be easy to test (somehow the numbers 6kb and 20kb keep popping up in my head, but I never tested it myself).

Things to keep in mind;
If you have to share data between the lua states, then some sort of synchronization must be implemented. The same holds for providing your own custom c functions in Lua and using them, then you also must ensure on the c side that there is proper synchronization implemented. But these are both common sense I suppose.

Hth
Thijs

> Many thanks for your kind help.
> Austin
> 
> 
> 
> 
> 
> 
> 
> 
> On Tue, Jun 24, 2014 at 4:33 PM, Thijs Schreijer <thijs@thijsschreijer.nl>
> wrote:
> 
> 
> > -----Original Message-----
> > From: lua-l-bounces@lists.lua.org [mailto:lua-l-bounces@lists.lua.org] On
> > Behalf Of Austin Einter
> > Sent: dinsdag 24 juni 2014 11:50
> > To: Lua mailing list
> > Subject: Re: Simple Function Call From C language
> >
> > Hi Thijs Schreijer
> > Basically I am using lua related functions such as
> > 1. luaL_checkinteger(L, -1);
> > 2. lua_pushinteger(L,4);
> > in my C code.
> > Once I call pushInteger api, do I need to call corresponding popInteger
> (or
> > similar) api so that stack in Lua or in C will be in tact. For me there
> will
> > be 1000s of processes and any process can call any Lua specific apis
> > randomly. So bit worried if I am messing up a bit with stack.
> You can only acccess a lua state from a single thread. So assuming you stay
> single threaded, then for each invocation you can simply clear the stack,
> and do your thing. Nobody will interfere with that.
> 
> >
> > One more question is, suppose a C program is running for 1 hour.
> Frequently
> > Lua scripts will be executed from C code.
> > Is it a good idea to do luaL_openlibs once at C code init time and
> lua_close
> > at C code exit time.
> > Or whenever I want to execute a script, do luaL_openlibs , then execute
> Lua
> > api, and then do lua_close.
> > If I have 10 scripts to execute, do 10 times luaL_openlibs and lua_close.
> >
> > Which approach will be better.
> This is a sandboxing question. Will users enter code, might it be malicious?
> Then you need tight controls, for example, some of the functions in the
> debug library you would better remove then.
> 
> Your first approach would be my favourite, but let each script run in its
> own environment that can be cleaned up by the GC after it finishes.
> Performance wise I think this would be best.
> 
> Thijs
> 
> > Thanks
> > Austin
> >
> >