RE: Is Lua the next big thing?

["Grellier, Thierry" <t-grellier@...>]

>> So when like us we need to share one lua_state between C++ threads,
but
>> with giving to the lua code the possibility to call a C++ wait() and
not
>> lua_yield(), we are in trouble. And python can here fill the gap.
>>   
>I still find this a little surprising, though.  Why can't you just use 
>yield on this?

I'm using SystemC a framework with offers kind of Hardware Description
Language. It is a basically a C++ library providing concept of
sc_process (implemented with coroutines), inside sc_module (implemented
as a class). 
sc_processes can do a blocking wait on sc_event. While doing so they let
the systemc library rescheduling them. That is, it resumes them once
event is notified. I've simplified a little, but these are roughly the
concepts.
What I'd like to do it to implement, for instance the sc_processes
inside a module with Lua code. But belonging to the same module, I'd
like them to share the module state, so basically I'd like them to run
in the same lua_State*. However I would like them to perform waits.
These wait are not lua_yield(), but the blocking wait of SystemC.

Thinking back of this, I realize now that is much simpler than I
thought, because I was focused on using lua_resume() and lua_yield()
while it was not needed at all.. If I associate a derived lua_State
created with lua_newthread to a sc_process, I've done all what I need!
The sc_process resumes the lua local context which then can call
sc_wait() and be blocked on it, it affects only local lua_State. If the
systemc schedules another lua/sc_process, it is in another local
lua_State, but that still share the global lua_State.
 
I was actually kind of competing with a veteran python programmer and at
this particular time he had much more free time than I had to work on
this, and I was still in my learning curve for Lua. He was also strongly
motivated keeping python in use there, instead of learning new stuff.

So I must correct myself on what I said it was not possible to do what I
wanted with lua, but I didn't see how to do it in the allocated time.
Maybe lua gems or pil2 would have helped to find a solution faster.
 
So I think the limit of extensive usage Lua, is that the organizations,
prone to use it, may already use another script language and thus may
not be very eager to migrate.


-----Original Message-----
From: lua-bounces@bazar2.conectiva.com.br
[mailto:lua-bounces@bazar2.conectiva.com.br] On Behalf Of Johann
Hibschman
Sent: Saturday, March 10, 2007 4:48 PM
To: Lua list
Subject: Re: Is Lua the next big thing?

Grellier, Thierry wrote:
> We are running C++ applications with both Lua an Python embedded. 
> Lua was initially attractive to replace the configuration text file we
> were using. Then we wanted to define through script the behavior of
some
> C++ application threads (note this are not real threads but
coroutines).
>   
That's funny.  I've been experimenting with Lua over Python just because

of python's global interpreter lock; using Python would limit me to 
having just one active OS thread.  I was all set to emphasize this point

more when I noticed that your threads "are not real threads but 
coroutines".  Good thing I checked.  :-)

> We haven't seen an easy solution with Lua:
> despite it offers coroutines, it doesn't allow to redefine lua_yield()
> to have an external control on the lua_resume(). So we were not
> confident in having C++ yields combined with lua_resume().
>   

I thought people were doing this all the time with Lua, but I've not 
done it myself.  Is it really this tricky?
> Whereas Python creates a state but offer low level primitives to store
> the interpreter state and restore it with a python context switch.
> Once python is embedded, this is not any more a pain to embed it.
Would it be possible to graft these primitives onto Lua?
> I guess that dealing with threads isn't an uncommon. With question
like:
> should I create one lua_State per C++-threads (then isolated, and only
> sharing the C++ layer).
> or share one lua_State between my C++threads, but then I must let the
> interpreter schedule them.
> or use pure lua coroutine but only lua can schedule the system.
>   
Aha.  See, now you're talking about real OS threads.  Here, it seems 
like Lua is the clear winner, due to Python's GIL.  I have a pool of 
worker threads, each with its own lua_State, and this works pretty well.

Now, I don't need much in the way of inter-thread communication; most of

the communication is of the "here, now do this, and give me back the 
answer when you're done" variety.

On, say, an 8-core linux box, using OS threads runs a lot faster than a 
single thread with coroutines would.
> So when like us we need to share one lua_state between C++ threads,
but
> with giving to the lua code the possibility to call a C++ wait() and
not
> lua_yield(), we are in trouble. And python can here fill the gap.
>   
I still find this a little surprising, though.  Why can't you just use 
yield on this?

-Johann