[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: Functions and memory usage
- From: "terenctb" <terenctb@...>
- Date: Fri, 18 Oct 2002 05:45:00 -0000
I have been thinking of doing something simliar to what Alberto has
been describing(we already do something like this for our other
memory manager) but I am not sure that it will be enough to cut down
the amount of data being accessed because of the method that the
allocation/gc is being done already does lend it self to speed.
The way I was going to do it was to just throw memory at it(since I
am in eviable position of using LUA on server with alot of
memory)..taking it from a pre-allocated area and sizing it so that
each allocatable area is tuned to the largest size possible. This
means no best fit, no frag and hopefully big boost in speed. BUT I
put some tracers on the way LUA alllocates/deallocates memory and it
seems a better way would be just to predefine a memory 'pool' for
each script..since each script is basically running the 'same' logic,
pre-allocation of all necessary data would faster..
Hmmm..now if only I figure out how to do it the short time I have;-)
Do you think the new version of LUA will have multiple 'memory'
managers instead of 1, cause I think right now this is the biggest
thing that people always end up changing in lua.
--- In lua-l@y..., "Alberto Demichelis" <alberto@c...> wrote:
> > What happens when you deallocate/free an object?
> Instead of keeping track of the pages we keep a linked list of feer
blocks for every chunk size,
> because they are "empty" we use the block himself as node of the
linked list so no memory overhead.
> When we create a page, the address of the first chunk is aligned in
a way that allow us to
> retrieve the page start address from the chunk address.The page is
ref counted(first 4 bytes).
> When we allocate a chunk we pop it from the free list and add a
reference to the page.
> When we free we put it back in the free list and we decrement the
reference if is 0 we free the page.
> I've took a quick look at dlmalloc, it looks not so far from our
approach exept for the implementation.
> I'll test it.
> PS: btw 200k/300k are internal structures + "waste".
> Alberto Demichelis
> Crytek Studios GmbH
> -----Original Message-----
> From: Nick Trout [mailto:ntrout@r...]
> Sent: Freitag, 18. Oktober 2002 01:06
> To: Multiple recipients of list
> Subject: RE: Functions and memory usage
> > 200k/300k of internal structures of the allocator on 100Mb of
> > allocated memory sound
> > pretty good to me, I do not understand the smile :-)
> Then its not memory wasted, its used to manage the memory
> assumed you meant that 200k-300k is lost to spaces at the end of
> and rounding up, as you describe.
> The smile was to show that while I might be aiming a criticism at
> there is no malice intended! Its all a bit dry this technical mumbo
> jumbo sometimes and email can often be read ambiguously.
> > > I assume each pool size would have an most recently used stack,
> > > allocating a new pool once you've checked the existing
> > pools for space?
> > I don't really understand the question :P
> > What do you mean as most recently used stack?
> What happens when you deallocate/free an object? Lets say you
> 10 pools (4k blocks) of 16 byte objects and then free a load of
> you then wanted to allocate a new 16 byte object, how would that be
> done? I assume for fast allocation you would keep track of pools,
> group them by size of object, so you could quickly find a pool
> containing 16 byte objects to allocate from. I was speculating that
> set of stacks of pools might be a good way to organise this. When a
> pool of a certain size is required it is pushed onto the stack and
> be accessed quickly. However when deallocation takes place this
> not be the best idea. I'd be interested to know how this is managed
> since you mention your free time was greatly reduced.
> If you are allocating 100Mb then this would be beyond the scope of
> certain platforms and so a system like yours may not be as
> suitable for all applications. The dlmalloc system may be a better
> general solution?