Re: "Random" delays in execution

["Jerome Vuarand" <jerome.vuarand@...>]

2008/10/23 Max van Rooij (van Rooij Electronics Design & Software
Engineering) <mvrooij@vanrooij-edse.nl>:
> A follow up.
>
> I already tested with only a few lines of code, just enough to produce
> the problem.
>
> What I have so far:
>
> - I allocate the 2D array in the main Lua _state_
> - I then create a new Lua _thread_
> - thread runs either until yield (which it has one)
> - then I perform a lua_resume (thread now runs to end
> - re-create a thread
> - start it again by lua_resume
> - repeat for ever.
>
> During each cycle I monitor memory usage. I notice that on each yield,
> the actual amount of mem used increases. After a few times it reaches
> somewhere between 150 to 200% of its initial memusage and the GC runs.
> With the 2D array of userdata there was relativly a lot of memory usage
> (start 492kb and just before a GC cycle 700kb...750kb). With same array,
> but not userdata but an integer, I have the exact same behaviour, but
> the numbers and delays are so small, I simply didn't notice at first....
>
> So, I now force a GC cycle whenever the thread returns LUA_YIELD as a
> result. Even with the "big" 2D array of userdata the problem is gone,
> e.g. no increasing mem usage and constant cycle times at the expense of
> more time / cycle.
>
> I'm almost happy now. The thing that bothers me, is the increasing lua
> memory usage per yield. Why is this? Is somehow data copied from lua
> _state_ to lua _thread_ and at threads end marked for GC? The thread
> itself first simply calls corotine.yield and then ends. No nothing any
> further. The 2D array in lua _state_ just lingers unused.

The coroutine itself takes some memory. So each time you terminate a
thread you are releasing that memory, but it will stay allocated until
a collection cycle. The memory includes the thread itself, its main
closure, any local/upvalue referenced by that closure, and the thread
global environment if it's not shared.

However only data recreated with each closure can affect the memory
usage growth. So if switching from userdata to integer for some
datatypes increase memory consumption pace, it means that such data is
allocated with each thread.