[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: Table Garbage Collection
- From: Rici Lake <lua@...>
- Date: Thu, 23 Nov 2006 12:28:10 -0500
On 22-Nov-06, at 9:06 PM, Glenn Maynard wrote:
On Wed, Nov 22, 2006 at 05:18:42PM -0800, Sam Roberts wrote:
On the other hand, maybe it doesn't matter that the heap is large,
pages will get swapped out, and if you never touch them, who cares?
You're going to touch them the next time you allocate more memory, and
the swapper--not knowing that memory was stale--will swap the old data
back in; whatever the user is doing that needs that memory will be
blocked while that happens. I care if applications are wasting my
system's performance by increasing swap file activity.
Some Unix systems have a way of telling the kernel that memory is
stale, through the madvise or posix_madvise interface. However, the
only Unix I work with a lot that really implements this feature in a
way that malloc() can make use of it is FreeBSD, which provides the
MADV_FREE advice option to madvise. Linux, as I understand it, will
interpret MADV_DONTNEED to mean that a page of zeros can be provided
instead of swapping back in, if the page was originally allocated
through mmap with anonymous backing store.
The FreeBSD standard malloc() implementation can be configured to use
this feature but it is rarely a win; although it does reduce swapping,
it also increases overall overhead because of increased bookkeeping on
In my experience, swapping out is more expensive than swapping in; in
the case where large regions can be marked stale, the FreeBSD facility
can speed things up quite a bit, but that is not very common.
glibc malloc()'s can be configured to return memory to the operating
system by providing a negative argument to sbrk(); this can only be
done if the memory to be returned is at the end of the heap. See
mallopt() for details (there's no manpage for it in the Linux system I
have access to right now, but it is documented on the glibc info pages
By the way, it might be possible to improve Lua's memory use by having
the allocation function use a C++ allocator. Knowing the size on
deallocation can give better characteristics than malloc/free can do,
especially for small allocations (eg. 4-byte allocations that really
4 bytes). I don't know if it would have an appreciable impact on total
memory use by Lua.
FreeBSD's standard malloc does not store allocation sizes in the
allocated block; all allocations on a given page are the same size.
(That's slightly imprecise, the details are available in the
documentation.) This can be a big win, but it also can be a memory hog;
it means that 40-byte allocations occupy 64 bytes of memory, instead of