lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

On 21-Dec-04, at 12:19 PM, Cory Bloyd wrote:

I'm worried about the malloc/free operations that happen as the internal
representation of the table changes.  If the malloc fails, does Lua try
calling the garbage collector or does it just blow up?  We will be
always running Lua in as-tight-as-possible memory pools so we will be
seeing allocation failures as a regular occurrence in the development

It "just blows up". Actually, it throws an error, which can be caught
with pcall; a subsequent garbage collection may well free enough memory
to continue.

I don't have a textbook definition handy, but "memory fragmentation"
refers to a problem that occurs in heap-style memory systems with
certain allocation/deallocation usage patterns.

How much this matters depends on the precise memory allocation patterns.
However, it seems likely that (non-incremental) garbage collection would
lead to less fragmentation in many cases, because memory is freed in bulk. Copying garbage collectors can completely eliminate fragmentation, but at a certain cost in headspace (that is, a trivial copying garbage collector
would guarantee that half of the memory is always usable, but you can do
better, particularly if you can play with virtual memory mapping.)
However, the Lua API is not particularly amenable to copying garbage

There are three clear usage patterns I see for growing a table.
1) Grow the table as an array indexed by sequential integers to take
advantage of the array optimization.  Eventually there will be too many
items to fit in the currently allocated array.  The system will need to
allocate a new array, copy the old values and then delete the old array.
Memory is fragmented.

Lua actually grows tables in this fashion, doubling the size of the
relevant part of the array every time. It uses realloc, which might
mean that the memory block is simply extended, depending on what other
malloc'ing is going on at the time. This strategy, which is a pretty
standard technique, works well with most malloc implementations, since
they tend to be optimised for this strategy (since it is pretty standard).

I realize I am being very picky about a problem that most people don't
have.  If I absolutely must have 100% memory safety I should just use a
less dynamic language.  Fortunately, my requirements are not 100%.  So
what I am trying to do is learn how to use Lua as well as possible so
that I can at least know what the rules are when I bend/break them.

Rule 1: don't try to second guess memory management until you have
proven that you have a problem. In the majority of cases, custom
memory management strategies are worse than well-tuned general
memory management systems.