This example is "blocking" all threads, but does not use recursion or reentrance.
The real problematic example is this one:
local mt;
mt = {__gc=function(o) setmetatable(o,mt) end}
do
local x={}
setmetatable(x, mt)
end
After the "end", "x" should be finalized but will never be finalized and we'll enter an infinite loop of gc cycles where "x" will be always present in the finalization list, so it will remain resident as if it was static. Now imagine the effect of:
while true do
local x={}
setmetatable(x, mt)
end
It will allocate an infinite number of new objects that will never be finalized. This thread will exhaust rapidly all resources allowed by the OS for the process, possibly allocating gigabytes or more that will never be used and will constantly grow the finalization list. The CPU will be running at 100%, with lot of paging to disk, causing severe problems to the host OS if the Lua host process is not given hard limits using the OS API (if not, the OS will finally hang and probably crash: this gives then a successful DOS attack). But if we can force a __gc to not finalize the object immediately by putting it in a "delay" list that will be marked as noit finalizable and will remain reachable, we can limit the frequency for finalization of these objects.
We still need a way to restrict the resources allocated by each thread/coroutine (and that's why a suggest a new "__alloc" metamethod for all Lua objects (and especially for threads created from a coordinating parent thread), so they cannot exhaust what is allowed and cannot freeze/crash the other threads or their host process or the whole host OS.
On Tue, Nov 27, 2018 at 10:30 AM Soni L. <fakedme@gmail.com> wrote:
>
> I am concerned about an attacker setting a __gc metamethod that loops forever and can't be broken.
How a recursive/renetrant GC can solve this issue? The first example I
can think of a "__gc metamethod that loops forever and can't be
broken" is:
```
local x = {}
setmetatable({},{__gc=fucntion()
while true do
x[#x+1] = {} -- Not necessary, but funny
end
end}
```
