Re: Memory leak?

["Mark Meijer" <meijer78@...>]

On 30/01/2008, alex.mania@iinet.net.au <alex.mania@iinet.net.au> wrote:
> >To make a bit more clear, in the following example, is what the string
> >says correct?
> >
> >foo = {}
> >bar = {"This string is never cleaned up."};
> >foo[bar] = nil;
> >bar = nil;
> >while(true) do end
>
> From my understanding, yes (Mike Pall?)
> I believe this would be sufficient too:
>
>  bar = "This string is never cleaned up."
>  foo[bar] = nil
>  bar = nil
>  while(true) do end

That would be sufficient, the way I currently understand it. But what
I was getting at, is the fact that (in my example) the entire table
contents (of the bar table) will remain in memory, simply because that
table (bar) has once briefly been used as a key in another table
(foo). In this case, the "entire table contents" is just a single
string, but what if it was more than that? If Lua's garbage collector
considers bar as being referenced for as long as that key is
"remembered" by the foo table, even after assigning nil to that key,
then the "problem" becomes a bit more pronounced. Don't know if that
is how the garbage collector sees it, but I suspect so.



> >Does that mean that Alex's trick above will only work sometimes,
> >depending on what keys are already in the hash part?
> Exactly, it'll only work if a resize of the hash table is required. When it's
> resizing the hash table, it resizes the array component at the same time. Note,
> if a table is used exclusively as an array then any setting of its hash table
> will require a resize (that's why the example works). (I didn't realize this at
> the time though =)

Ok that's good to know. So basically, for any table that has, or has
ever had (lol), non-integer keys... there is currently no guaranteed
way of reclaiming the memory of "remembered" keys?  (unless any and
all non-integer key/s it ever had was/were only ever assigned nil) ...
is that right?

(btw we need a better term than "remembered keys" :P...)



> Hmm, I guess it would involve setting a flag every time "next" is called, and
> clearing it whenever a new key is created. When the table gets marked during a
> garbage collector cycle, resize vectors if they're disproportionately large.
> Wouldn't slow down tables measurably I don't think.. nor the gc (whenever a table
> is marked, the incremental gc takes in to account how expensive the mark
> operation was... could take any resizing operations in to account).
>
> Good luck finding a spare bit in the Table structure though =). Also, it may
> produce intermittent errors in "next" in programs that currently get away with
> modifying the table. (They shouldn't, but you still can get away with it on
> occasion). There's also the question of how many programs would actually benefit
> from it.

There is that question. That's also why I wouldn't want to propose
anything too involving.

But correct me if I'm wrong in assuming that next() only benefits from
these "remembered keys" during table traversal. This should have no
impact on the implementation of something like collectgarbage("keys").
No need for flags and whatnot. As long as it's clear (i.e. documented)
what it does, sothat nobody would get the idea that they can and
should call collectgarbage("keys") *during* table traversal...

(in which case the same thing happens as the table resize that is
currently triggerable only by adding a new item, which is the reason
why the behaviour of next() is undefined in that case, I imagine...
and which is why *that* is documented).

Right now, there is some code somewhere in the Lua core that does the
equivalent of this (pseudo-code):

------------------------------------
function TableAddItem(SomeTable, SomeKey, SomeItem)
	if TotallyNewKeyThatsNeverBeenUsedBefore(SomeTable, SomeKey) then
		TriggerTableResize(SomeTable);
	end
	... etc
end
------------------------------------

All I'm suggesting is something equivalent to the following to be
added (pseudo-code again):

------------------------------------
function CollectGarbage(StringyThingy) -- Existing function
	if StringyThingy == "count" then -- Nothing new...
		return GCWalkHeap();
	elseif StringyThingy == "collect" then -- ... still nothing new...
		GCDoFullCycle();
	elseif StringyThingy == "keys" then   -- here we go...
		TriggerTableResize(SomeTable);
	elseif ... etc
	end
end
------------------------------------