[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: Heap usage and variable argument C functions
- From: Rici Lake <lua@...>
- Date: Tue, 21 Dec 2004 12:45:48 -0500
On 21-Dec-04, at 12:19 PM, Cory Bloyd wrote:
I'm worried about the malloc/free operations that happen as the
representation of the table changes. If the malloc fails, does Lua try
calling the garbage collector or does it just blow up? We will be
always running Lua in as-tight-as-possible memory pools so we will be
seeing allocation failures as a regular occurrence in the development
It "just blows up". Actually, it throws an error, which can be caught
with pcall; a subsequent garbage collection may well free enough memory
I don't have a textbook definition handy, but "memory fragmentation"
refers to a problem that occurs in heap-style memory systems with
certain allocation/deallocation usage patterns.
How much this matters depends on the precise memory allocation patterns.
However, it seems likely that (non-incremental) garbage collection would
lead to less fragmentation in many cases, because memory is freed in
Copying garbage collectors can completely eliminate fragmentation, but
a certain cost in headspace (that is, a trivial copying garbage
would guarantee that half of the memory is always usable, but you can do
better, particularly if you can play with virtual memory mapping.)
However, the Lua API is not particularly amenable to copying garbage
There are three clear usage patterns I see for growing a table.
1) Grow the table as an array indexed by sequential integers to take
advantage of the array optimization. Eventually there will be too many
items to fit in the currently allocated array. The system will need to
allocate a new array, copy the old values and then delete the old
Memory is fragmented.
Lua actually grows tables in this fashion, doubling the size of the
relevant part of the array every time. It uses realloc, which might
mean that the memory block is simply extended, depending on what other
malloc'ing is going on at the time. This strategy, which is a pretty
standard technique, works well with most malloc implementations, since
they tend to be optimised for this strategy (since it is pretty
I realize I am being very picky about a problem that most people don't
have. If I absolutely must have 100% memory safety I should just use a
less dynamic language. Fortunately, my requirements are not 100%. So
what I am trying to do is learn how to use Lua as well as possible so
that I can at least know what the rules are when I bend/break them.
Rule 1: don't try to second guess memory management until you have
proven that you have a problem. In the majority of cases, custom
memory management strategies are worse than well-tuned general
memory management systems.