Re: GC and userdata objects

[Dibyendu Majumdar <mobile@...>]

On 12 April 2018 at 07:58, 云风 Cloud Wu <cloudwu@gmail.com> wrote:
> Dibyendu Majumdar <mobile@majumdar.org.uk>于2018年4月12日周四 下午1:32写道：
>
>>
>> The scenario is this - I am creating a AST generator for Lua; the AST
>> is captured in a userdata. The AST is generated every time some code
>> is parsed - so you can imagine that this can be quite frequent. Say in
>> a test program, many code chunks are parsed resulting in several
>> userdata objects of different sizes being created. I noticed that the
>> GC does not collect the objects fast enough. In this scenario, how do
>> you suggest I should use the GC step parameter? Keep setting it to
>> different values based on userdata sizes? Or keep track of total
>> userdata memory and set this accordingly? Obviously the rate of
>> allocations varies over time, and sizes vary as well.
>
>
> I don't think make GC more aggressive by the actual memory size annotation
> would be helpful in this scenario.
>
> My solution is putting all  AST in one C container . Userdata only stores an
> ID that associate to the AST, and reference the code (string or function) in
> a weak table.
>

At the moment I am putting all nodes for a single chunk into one C
container; so:

x = ast.parse 'return'

Creates a fulluserdata object that has all the AST for that chunk.

But if I do:

y = ast.parse 'if x then return 42 end'

This creates a separate fulluserdata object separate from the one
before. Again all the real structure is inside the C data structure.

Are you saying that the both of these should live in the same C data structure?


> And then we can remove some older  AST  (by LRU) every time new code is
> parsed, If we need an AST already removed, just rebuild from code again.
>
> The C container of AST is only a cache, so we can also clear the cache when
> we need more memory.
>

I suppose I could also have a method ast:release() that gets rid of
the underlying C data structure; this can be used when the code no
longer needs the AST.


Regards
Dibyendu