Re: Share proto object among multiple lua vm

[Coroutines <coroutines@...>]

On Thu, Mar 20, 2014 at 7:55 AM, Coroutines <coroutines@gmail.com> wrote:
> On Thu, Mar 20, 2014 at 6:42 AM, 云风 <cloudwu@gmail.com> wrote:
>> 2014-03-20 20:51 GMT+08:00 Roberto Ierusalimschy <roberto@inf.puc-rio.br>:
>>
>>> Are you really sharing the Proto object or only the bytecode (and maybe
>>> some other fields)? If you are sharing the Proto object, how to you
>>> handle its constants (field 'k')? They include strings, and short strings
>>> cannot be shared (at least directly), due to internalization.
>>>
>>> -- Roberto
>>>
>>
>> I wish lua can support share strings and prototypes directly in future
>> version , so that we can run many lua states on many-core , and
>> passing the constants between lua states can be very cheap (like
>> Erlang does).
>>
>> --
>> http://blog.codingnow.com
>>
>
> I personally wish it were easy to exchange data between lua
> global_State's.  The main lua state -- as I understand it -- contains
> both a data portion and an execution context.  When you create a
> coroutine/thread you are creating an execution context, but its data
> portion references the data portion of the main 'global_State'.  So
> when you index a table it fetches it from the data structures in the
> global_State, and the main lua state can independently fetch its own
> data structures from that same area of memory.  The problem when
> trying to share process memory between several threads is each thread
> can modify the data portion of that main state independently and
> corrupt it by doing so without coordination.  You could modify a table
> concurrently while another state is trying to access it and well...
> let's just say unexpected things happen (in theory).  I hate locking
> the main global_State, I wish it were easy to exchange data between
> global_States without serializing into a string :(  Maybe I have this
> all wrong.  My 5.3 wish, though. :-)
>
> PS: You can use lua_xmove() to exchange data between the execution
> context portion of a lua_State but you can't easily move data between
> global_States (from one data portion to another.  It's not really
> something covered in the PiL, haha.  I imagine it would be quite
> expensive and would involve invoking the GC in many cases to clean up
> data that is officially "leaving" the originating global_State.

Hmm, I think I have it wrong, all lua_State's reference the
global_State.  I want to be able to easily exchange things without
marshalling between global_States of independent Lua processes.  When
you retrieve something from _G it pulls it from the global_State into
the lua_State onto that virtual stack (I think), and it's easy to
exchange data between lua_State's -- but if you want to exchange data
between independent processes/address spaces it gets difficult because
you have to resort to marshalling/serializing things. :(  I think you
could host 2 Lua "instances" within the same shared address space and
pull something into 1 lua_State, then exchange it to the other Lua
instance with xmove() to that lua_State, and then store it in the
global_State there by assigning it into _G.. I'm not sure.  There
might be unseen references somewhere that make this dangerous.  I tend
to talk to myself a lot ~

If there were an API for exchanging data between global_State's you
could do it rawly through a communication channel (a shared memory
segment?), and there'd be no marshalling overhead :>

Food for thought: http://www.lua.org/source/5.2/lstate.h.html#global_State