lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]
There does not seem to be a splendid work-around solution. The best I can come up with is using shared memory (man shmget) to get the two threads (and Lua states) of the same process talk to each other. All I really need to pass is the value of _one_ pointer (or NULL if no- one's been there). But there's no mechanism for that, since normally globals are globals. Now, they're not. 

shmget:
	+ it would work
- creating a shared memory area asks for "unique" key number. no way to guarantee any uniqueness... - _any_ processes on the machine are welcome to trash us, if they use that "unique" number 

file system (create /tmp/myptr)
	+ it would work
- trashing file system, no "unique" name (that unrelated threads would know)
If we turn the case upside-down, and think "what if" PRIVATE flag was not used in Lua core. 

o Multiple process, multiple Lua states:
	no effect (having their memory areas completely separate)

o Single process, single Lua state:
	no effects?
what if two modules have the same global variable name? does someone know what happens? 

o Single process, multiple states (multicore case):
globals in a module would share addresses, thus being able to do "inter-state" communications (if needed) Thread Local Storage can be used, in case modules like to keep things private (but need to explicitly do so) also "static" keywords seems to imply by-thread behaviour, but I wouldn't trust that much
So, should I bang my head, make a <100% safe hack, or just call this a quit and try to lobby for ditching PRIVATE in Lua in >= 5.2 ?
And.. are other systems affected? If they don't have the concept of dll/so-private globals, there's not much benefit having OS X ask for it? 

-asko


Asko Kauppi kirjoitti 3.4.2007 kello 13:08:



Thanks for a good description of the issue.
So, I take it that the PRIVATE behaviour is expected, "by design" and not by chance. It kind of does make sense, since privacy is always good, and using separate states has been a marginal issue. WIth multicore processors, this may change, though. 

Some points:
- is this behaviour consistent over architectures (Linux, Windows, ...) - is it documented, because it sure is an important point for module craftsmen
I may need to use named shared memory, or something, to actually get the module instances know each other. Given that the PRIVATE behaviour is there to stay. It may still be best to have it like this, requiring people who want the instances to know each other to take the extra steps. We may need a wiki page on this? 

-asko
ps. in order to be at the same addresses, the variables need to be global (not static) even if the PRIVATE flag is not there. This would give a neat way of balancing between the two "modes" but maybe it's too subtle, and a gcc upgrade would blow it off, some day? :) 



Jerome Vuarand kirjoitti 2.4.2007 kello 23:10:


Sam Roberts wrote:

On Mon, Apr 02, 2007 at 09:49:56PM +0300, Asko Kauppi wrote:
Lua uses NSLINKMODULE_OPTION_PRIVATE flag on OS X for its "require", 
but this causes globals in a module required in two separate Lua
states NOT to have common globals. I faced this today, and the only
solution found was removing that flag from loadlib.c.


As a quick fix, have you tried using dlopen() instead? It is
recommended on OS X (by Apple, see
http://lua-users.org/wiki/BuildingLua).

Still, I'm a little surprised that the bundle was loaded twice.
Could this be a bug? Isn't package supposed to load the bundle/so/ dll 
only the first time the module is required, and the other times to
use the previously opened shared library handle, find the luaopen_XX
symbol using that handle, and call it again in the new state?

The package module definitely keeps the load handle around, I don't
see why it shouldn't use it!
The problem comes from the fact that Asko use separate Lua states. If a 
binary module is required in two independents Lua states, it's
initialization function must be called twice. Since binary modules may 
not be reentrant and use globals to store "Lua-state"-specific
information, having it loaded twice with private static data allows to 
avoid collisions in memory between the two instances of the module,
while the package system still ensure that all require in the same Lua 
state will always return the same module table.

If you make the static data shared (by removing the aforementioned
flag), to allow communication between the two instances of the module, 
you have to make sure all binary modules you load don't store
"Lua-state"-specific information in static variables. For example you
can't store registry indices in globals and share them between Lua
states, since each has its own registry.