Re: luaproc

[Asko Kauppi <askok@...>]

Matthew Wild kirjoitti 19.2.2009 kello 15:08:

> On Thu, Feb 19, 2009 at 7:58 AM, Asko Kauppi <askok@dnainternet.net>  
> wrote:
> > luaproc really seems interesting.
>
> Agreed. Finally I have something to say to people who say I should be
> writing my project in erlang :)
>
> > The merger of coroutines (yield) and actual parallel threading is  
> > smooth.
> > Maybe we can get a new threading model in Lua 5.2 (or 6.0) that  
> > will merge
> > them even further?
> >
> > Mapping Lua states M:N with OS threads is the right way to go. I  
> > shall adopt
> > that to Lanes, as well.
>
> That's good to know. The only reason I'm hesitating about Lanes at the
> moment is that it is restricted to one machine. Transparently
> communicating with "processes" running on a remote machine would be
> great, but I'm not sure this is a direction Lanes is headed in (and I
> quite understand if it isn't).

Lanes could be extended to work together with LuaSocket, to have  
network multithreading. Just as the concept of luaproc + LuaSocket has  
been made. It would, however, require serialization, which currently  
is (no more) needed in Lanes. And it would need some testing. If  
someone does all that, I'm happy to bring it in, but I think it could  
be kind of outside of Lanes itself.

Another such message passing media is DBUS; networking is not the only  
one. It would make a lot of sense to have a DBUS binding and Lanes  
work nicely together with each other.

> > What I see as downsides of luaproc is mainly the restriction of  
> > message
> > types (number, string, boolean). This can be overcome by other  
> > addon modules
> > that would, for example, implement Linda shared data storage s.a.  
> > what Lanes
> > has. It need not be part of luaproc itself, though.
>
> And this a reason I couldn't (yet) use luaproc. It would be great to
> see a solution such as you describe, with as low an overhead as
> possible on transfers of other types (primarily tables).

Lanes uses a hidden, interim state for data passes. This is more  
efficient than serialization, at least in most cases.

What is interesting regarding luaproc is that since its message  
passing is synchronous, I think the intermediate state could be  
skipped. Data could be lifted directly from state A to state B. Not  
only strings, numbers and booleans, but any data (and closures). Lanes  
needs the interim state as a data store, and for synchronization.

> > I do hope luaproc (or a derivative) eventually becomes an integral  
> > part of
> > Lua, itself. Meaning it would be a package s.a. 'coroutine'  
> > currently is,
> > and accompany Lua in the source tgz.
>
> I suppose it wouldn't, at least as-is. ANSI C doesn't have threads :(
>
> That said, perhaps it could simply disable multiple workers when
> compiled on an unsupported platform, and work just as coroutines do
> today.

Exactly.  The couple of lines of pthread and Win32 API that would be  
needed are no more horrendous than current dlopen mentions. So what.

This all depends on how much push the authors want to have on good  
multicore performance. Out of the box.

> Things are getting interesting, great projects, both. :)

Thanks.

-asko


> Matthew.