 |
lua-l archive |
|
|
||
|
On Aug 7, 2015 10:28 AM, "Javier Guerra Giraldez" <javier@guerrag.com> wrote:
>
> On Fri, Aug 7, 2015 at 8:51 AM, Rena <hyperhacker@gmail.com> wrote:
> > Even Python, the "batteries, charger and solar cells included" language,
> > seems to have missed this boat - it provides "threading", but only one
> > thread can actually run at a time (global interpreter state is locked while
> > executing) and it doesn't sound like it'd be easy to fix without breaking
> > other things.
>
> that's why there's also the "multiprocessing" module, that uses
> multiple processes,each with their own interpreter stack and GIL
> (sound familiar?). It's been there for several years now, and it's
> usually regarding ad "the replacement" for threads.
Yes, I thought about that but decided not to mention it since it's running multiple processes, not multiple threads in a process. Still a useful tool, but not quite the same thing. No shared variables for example.
> and it's terribly inefficient: in multicore hardware, most simple
> locks are by nature system-wide, having to propagate to _every_ core
> in the system. By experience, even in a high-memory-bandwidth system
> like modern Xeon familes, there's a low maximum number of inter-core
> messages per second. Think like a core can do several thousand
> operations in the same time as propagating just one lock.
Ouch. It's really necessary to allocate a hardware lock every time? Not enough to use an atomic test-and-set instruction on a per-shared-object lock flag?
> > The thread libraries I've used run a separate Lua state in each thread and
> > provide some type of communication channel between them. This is a nice
> > simple model, but maybe not the most efficient.
>
> Au contraire, passing messages is far more efficient than sharing
> variables; because you only incur in one inter-process fault, instead
> of trashing caches on almost each access of shared variables.
Right, I meant the Lua implementations aren't very efficient, because nothing is actually shared between the states; to pass strings, the receiving state has to make its own copy (wasted memory) and hash it (wasted time) and the sender still has to garbage-collect its own copy (wasted time); to pass tables requires recursively copying each element (slow and error-prone - watch out for cycles, don't forget to copy the metatables too, and what do you do if one of them can't be copied due to upvalues?); to pass functions requires converting to a bytecode string and back (and some can't be copied); to pass userdata requires the underlying C code to be aware of the possibility (e.g. use reference counting outside of Lua to track how many states have an object - many libraries destroy the native object in the __gc metamethod, which won't work well if multiple states reference it).
If Lua provided some methods to deal with these cases, perhaps by allowing objects to be owned by multiple states, giving the __gc metamethod a "remaining ref count" parameter, letting states share memory blocks (especially for immutable objects such as strings) or even just providing a lua_xmove that can copy between independent states (which could probably be done more efficiently by a built-in function that can poke at the internals than by an external module), I'm sure a lot of the overhead could be eliminated.
> --
> Javier
>