Re: Memory leak?

[Hans Hagen <pragma@...>]

Mark Meijer wrote:

> Good points. I just figured it would all simply be garbage collected.
> To make a bit more clear, in the following example, is what the string
> says correct?
>
> foo = {}
> bar = {"This string is never cleaned up."};
> foo[bar] = nil;
> bar = nil;
> while(true) do end

i don't know the lua c course but this is what i expect

  foo = {}
  bar = {"This string is never cleaned up."}

both create a variable with a table structure as value and in the second 
case that table has a value (maybe some extra space allocated depending 
on the kind of table; the string is represented by a pointer

  foo[bar] = nil;

here you index foo with bar (its pointer representation i expect) but 
since you set it to nil no real entry is created (not an indexed table)

  bar = nil;

this frees bar's value (not the variable (unless local and not 
referenced); it also frees "This string ..", unless it's used elsewhere 
too)

  while(true) do end

this will loop -)

a variant:

foo = {}
bar = {"This string is never cleaned up."};
str = "This string is never cleaned up."
foo[bar] = true
bar = nil;

here "This is.." is still allocated (referenced by str) and bar's 
original table is also kept (a key in foo, which will pass by when you 
pair over it)

> Great :) At least there's a way around the problem. Building on the
> above, I have found that "foo.bar = nil" will consistently and
> repeatedly reclaim the memory used by "remembered" keys. Apparently,
> when assigning it nil instead of some other value, the "bar" key is
> reclaimed immediately as well, which allows the same trick to work
> again and again.

indeed a nice trick; i wonder what happens with sparsely indexed tables 
(i.e. when lua decides to index of hash or use a combination)

> > Tables expand automatically, but shrink only under certain
> > implementation-specific conditions. Under Lua 5.x this happens
> > only when resizing a table. This is triggered by inserting a key
> > which neither fits in the array nor the hash part.
>
> Does that mean that Alex's trick above will only work sometimes,
> depending on what keys are already in the hash part?

it would be nice if the lua documentation would explain this in more 
detail; my lua apps have hundreds of megabytes (complex) tables and i'd 
like to know where optimization can take place

> Obviously I didn't know about the conditions which cause that stuff to
> be cleaned up. I'm sure some thought went into that, because if it'd
> never get cleaned up at all then I'd say it would be a big problem.
> For any app that gets to see a lot of different keys in their
> lifetime, and/or usually runs for long periods of time. For example
> anything that does a lot of reverse-lookup/indexing, or caching or
> such things. The Kepler Project might not be possible. But yeah, the
> way it is, I guess it's not a major issue :)

for tracing purposes i keep track of memory usage and when presented by 
graphics it's quite interesting to see when the gb pops in (first a time 
consuming peak (sweep) and then successive calls)

-----------------------------------------------------------------
                                          Hans Hagen | PRAGMA ADE
              Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
     tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
                                             | www.pragma-pod.nl
-----------------------------------------------------------------