Re: GC and userdata objects

Top posting, since I am replying to the entire thread.

This discussion brings back to my memory some older discussions in the list, about deterministic collection/finalization. The principal issue is that some userdata are much more than a handful of bytes, they can be handles, connections, money and what not. I think there is a need to be able to mark a userdarum so that this particular userdatum (and everything that references it) becomes subject to very aggressive collection, perhaps with ref-counting. The argument "this is bad for performance" has been duly noted. I think it is possible to have two major branches in the VM interpreter, the regular branch that works like it does today (without any extra bookkeeping for any object), and the "aggressive" branch which is switched into when there is at least one userdatum marked as described above, and exited as soon as they are all gone. That way the price for the extra bookkeeping will only be paid by those who are willing to pay it.

Cheers,

On Thu, Apr 12, 2018 at 10:28 AM, Francisco Olarte <folarte@peoplecall.com> wrote:

On Wed, Apr 11, 2018 at 9:00 PM, Reinder Feenstra
<reinderfeenstra@gmail.com> wrote:
> A possible solution could be if you we're able to hint the GC a virtual size
> for the userdata.

Yeah, that may be nice. A single word can contain it, but you would
need to hint it right, if you have a forest you need to hint the
forest size, ....

> I'd had the same issue with calls to lua from C++ with userdata. The
> userdata only contained a std::shared_ptr, so it was small for lua, but it
> contained "virtually" a data chunk that could be a few MB up to 256 MB.
> Under some conditions up to 1000 of them where kept alive until the GC
> decided to clean up some of them. We "solved" it by making the GC very
> agressive, to make sure unused reference were cleaned up fast.
> I've not yet found a better solution.

And there comes another problem. I have many objects with shared ptr
to them stored in userdata, and then they contain a lot of shared ptr
internally.

Estimating the full size of reachable memory from each userdata would
lead the GC to think they consume many times more than they usually
do.

Also, there are shared ptrs unknown to lua, so even without external
sharing lua can overstimate the amount of memory it's going to free (
or the used one ), but this is probably harmless.

What I've found is in directly-controled memory, lua owned objects, I
can just allocate the chunk in the user data ( and slice it with
placement new if needed ), but for complex stuff I just do not have
enough control of how much would be freed, and it does not normally
matter anyway.

Francisco Olarte.