Re: GC and userdata objects

[Dibyendu Majumdar <mobile@...>]

On 12 April 2018 at 08:49, 云风 Cloud Wu <cloudwu@gmail.com> wrote:
> Dibyendu Majumdar <mobile@majumdar.org.uk>于2018年4月12日周四 下午3:18写道：
>>
>>
>> At the moment I am putting all nodes for a single chunk into one C
>> container; so:
>>
>> x = ast.parse 'return'
>>
>> Creates a fulluserdata object that has all the AST for that chunk.
>>
>> But if I do:
>>
>> y = ast.parse 'if x then return 42 end'
>>
>> This creates a separate fulluserdata object separate from the one
>> before. Again all the real structure is inside the C data structure.
>>
>> Are you saying that the both of these should live in the same C data
>> structure?
>
>
> I mean AST data structures can be rebuild from the code string, so the
> container is only a cache. Yes, both of these can live in one cache. And we
> can clear the cache at any time.
>
> x = ast.parse 'return'   make an userdata just [id:1] , and put
> [userdata:'return'] into a weak table. This userdata is a small one .
>
> The real AST is in the C AST cache structure like  [ 1:AST for 'return' ,
> ... ] .
>
> When we call some ast api later , remove some less used AST in this cache
> just like collect garbage. It's in dependent of lua gc , so it can run more
> aggressive.
>

Okay understood.

>> > And then we can remove some older  AST  (by LRU) every time new code is
>> > parsed, If we need an AST already removed, just rebuild from code again.
>> >
>> > The C container of AST is only a cache, so we can also clear the cache
>> > when
>> > we need more memory.
>> >
>>
>> I suppose I could also have a method ast:release() that gets rid of
>> the underlying C data structure; this can be used when the code no
>> longer needs the AST.
>
>
> Manually management by ast:release() can work, but use LRU algorithm to
> release automatically would be easier and less bug (IMHO) . If we hold some
> AST for later use, it's safe to release automatically first, and rebuild
> from code when use again.
>

In my case the ultimate goal is code generation - so the AST will most
likely be used immediately after and then never used again. A manual
release might be easiest option here.

But of course the AST can be used for other purposes too - for
instance by someone else that needs it ... in that case a different
strategy is necessary. But you are right I could cache the source
string / and AST as a pair - and just rebuild AST from source if
necessary. But I will need to hang on to the source until ... ?

Thanks and Regards
Dibyendu