Re: Lua 5.2 garbage collector

[Shawn Fox <shawnkfox@...>]

On Sat, Aug 14, 2010 at 11:07 PM, Drake Wilson <drake@begriffli.ch> wrote:

Quoth Shawn Fox <shawnkfox@gmail.com>, on 2010-08-14 20:31:38 -0500:

> Any chance of adding a way to set a memory cap?  This would be very useful
> for me and I'd think others who are building sandbox type environments would
> find it useful as well.  I'm using a custom memory allocator right now to
> cap memory usage, but it really ought to be part of the standard API, like
> lua_gc(L, LUA_GCMAXMEMORY, 67108864) or some such...

I'm a little suspicious.  It seems like the kind of convenience
function that would be hanging off the side of the rest of the GC
settings without actually belonging there.  I tend to think the
allocator hooks are good enough for that; besides, that would let you
do things like soft and hard limits of any form you like (so, free up
an emergency allocation for recovering when your initial cap runs out,
&c.), and then modify the behavior as necessary for the application.
I think a single hard memory cap isn't as useful as it might seem, and
anything more complicated would get into significant complexity for a
feature that's not very cohesive with the rest of Lua core.

Just one person's opinion though.  :-)

  ---> Drake Wilson

I think that capping memory is such a common requirement that it ought to be part of the standard Lua implementation.  I don't care if it is by calling lua_gc or some other method, the point is that I should just be able to set a hard cap for memory and cpu usage in a sandbox environment and Lua should just abort if it goes over that limit.

Considering that Lua is generally used as an embedded language the requirement to be able to cap memory and CPU usage seems to be something that would be built into the language.  I was actually quite surprised to find there wasn't an API call already in the language.

I'm using Lua (actually LuaJIT 2.0 beta4)  in a situation where users can submit arbitrary Lua code for execution in a massively parallel compute grid.  A single execution could be spread across 100s of systems and I'd like a good way to limit CPU and memory usage on any individual system.  The best case for me would be a new API call that allows me to specify a max memory and cpu usage, something like:

lua_pcall_limit(L, nargs, nresults, errorfunction, maxmemory, maxcpu);

I've got a system that works fairly well using lua_sethook and a custom memory allocator, but to me this just seems like it ought to be a native part of the language.