lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index] 


[Agglommerating multiple posts long after the event ;-]


I was rather hoping to handle a couple of hundred
proxied sockets inside a simple Lua program... whilst
there is plenty of physical memory to support this I
am, absurdly, running out of address space :-)

What's going on?



Each task requires a C stack; the default stack allocation
on linux is quite large.

You might want to try using plain old Lua coroutines,
which do not require a C stack but which do have some
limitations as a result.
The good news is that my failure to do this the "easy" way using LuaTasks (which are actually very cool -- kind of like fork()s from a separation point of view, but without the fork()ing overhead) means that I will almost certainly knuckle down and do it the "proper" way using coroutines.
It's my own fault for not pursuing the elegant solution in the first place :-) 


Adding:

pthread_attr_setstacksize( &attr, 2097152/16 ); // Thanks Asko !!!
before pthread_create in syncos.c line 62, my test runs using 42m of virtual memory on Ubuntu 7.04 

But... with more than 255 subtasks... SEG FAULT
Since this implies a LuaTask 1.6.1 some time in the not-too-distant future, how about adding an optional third param to task.create() allowing a stack size to be specified?
(You could add a new error code, say -5, for "stack size out of range" in case pthreads or _beginthreadex() doesn't like the size, e.g. it's too small, or just re-use error -4 in this case.)
For an OS implementation in which the stack size can't (easily) be set, or isn't needed, simply ignore the parameter quietly...
Just a thought. Would allow the stack size to be tuned for individual tasks, which would be in keeping with Lua's "don't be huge" attitude! OSes seem to vary quite a bit -- 64k on FreeBSD IIRC, 1m on Windows, 8m in Linux pthreads -- so some way of "saying what you mean" for each subtask would be entirely suitable IMO.
(BTW: about the segfault, which I blithely said was "unsurprising"...I assumed it was happening at around 250 because 256*8m = 2g, but your tests show it's not down to that. Anyway, Linux slices VM between user and kernel at 3g, doesn't it? Not the default 2g of Windows, which I am more familiar with. Oh, unless you run Exchange, which is such a beast that Windows offers you the special "/3G" bootup option to start kernel-land at 0xC0000000 to make room for all those pesky calendar items :-)