[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: LuaTask and memory allocation
- From: Asko Kauppi <askok@...>
- Date: Wed, 20 Jun 2007 18:29:58 +0300
You can actually limit the stack size also with some environment
variable (without touching code), maybe the RLIMIT_STACK that Taj
Daniel Quintela kirjoitti 20.6.2007 kello 15:40:
Asko Kauppi escribió:
Lua Lanes does the following to limit OS side stack usage.
I've tested what seems to be the minimum stack size on each OS
(they do overestimate, "just in case"). Of course, one might think
of a C binding that really needs huuuuuge stack size, but I doubt
it in action.
Lua itself uses mostly the heap so this should be safe.
#if (defined PLATFORM_WIN32) || (defined PLATFORM_WINCE)
#define _THREAD_STACK_SIZE 0
// TBD: does it work with less?
#elif (defined PLATFORM_OSX)
#define _THREAD_STACK_SIZE (524288/2) // 262144
// OS X: "make test" works on 65536 and even below
// "make perftest" works on >= 4*65536 == 262144 (not
#elif (defined PLATFORM_LINUX) && (defined __i386)
#define _THREAD_STACK_SIZE (2097152/16) // 131072
// Linux x86 (Ubuntu 7.04): "make perftest" works on /16
(not on /32)
#elif (defined PLATFORM_BSD) && (defined __i386)
#define _THREAD_STACK_SIZE (1048576/8) // 131072
// FreeBSD 6.2 SMP i386: ("gmake perftest" works on /8 (not
#if (defined _THREAD_STACK_SIZE) && (_THREAD_STACK_SIZE > 0)
PT_CALL( pthread_attr_setstacksize( a, _THREAD_STACK_SIZE ) );
Rici Lake kirjoitti 20.6.2007 kello 0:31:
On 19-Jun-07, at 4:21 PM, Paul Ducklin wrote:
Using LuaTask 1.6.0 statically linked into
Lua 5.1.2. (A few other packages are built-in,
namely the LuaSocket core, lfs, lrandom, bitlib
and struct.) This is on Linux.
Starting Lua, mem use (as shown by top) is about
1m resident, 2.2m allocated (virtual). Seems quite
modest to me.
Every time I call task.create() (even if the tasks
do nothing but sleep immediately) mem usage goes
up by about 70k resident, but a whopping 8m
of virtual memory. So with 250 tasks I am _using_
just 17m but have 2g allocated! Perhaps unsurprisingly,
at 257 tasks Lua segfaults...
Why 2m for Lua but an additional 8m of virtual memory
for each created task, esp. when the task is only
_using_ a modest 70k?
I was rather hoping to handle a couple of hundred
proxied sockets inside a simple Lua program... whilst
there is plenty of physical memory to support this I
am, absurdly, running out of address space :-)
What's going on?
Each task requires a C stack; the default stack allocation
on linux is quite large. There might be a way to reduce it;
I'm not al tanto with LuaTask. Other operating systems may
(by default) make a smaller C stack, but since C does not
check for stack overflow, that can be dangerous.
You might want to try using plain old Lua coroutines,
which do not require a C stack but which do have some
limitations as a result.
pthread_attr_setstacksize( &attr, 2097152/16 ); // Thanks Asko !!!
before pthread_create in syncos.c line 62, my test runs using 42m
of virtual memory on Ubuntu 7.04
My test is a main task with 255 subtasks blocked on task.receive( -
But... with more than 255 subtasks... SEG FAULT
I will try to found the bug asap...