At the moment I am putting all nodes for a single chunk into one C
container; so:
x = ast.parse 'return'
Creates a fulluserdata object that has all the AST for that chunk.
But if I do:
y = ast.parse 'if x then return 42 end'
This creates a separate fulluserdata object separate from the one
before. Again all the real structure is inside the C data structure.
Are you saying that the both of these should live in the same C data structure?
I mean AST data structures can be rebuild from the code string, so the container is only a cache. Yes, both of these can live in one cache. And we can clear the cache at any time.
x = ast.parse 'return' make an userdata just [id:1] , and put [userdata:'return'] into a weak table. This userdata is a small one .
The real AST is in the C AST cache structure like [ 1:AST for 'return' , ... ] .
When we call some ast api later , remove some less used AST in this cache just like collect garbage. It's in dependent of lua gc , so it can run more aggressive.
> And then we can remove some older AST (by LRU) every time new code is
> parsed, If we need an AST already removed, just rebuild from code again.
>
> The C container of AST is only a cache, so we can also clear the cache when
> we need more memory.
>
I suppose I could also have a method ast:release() that gets rid of
the underlying C data structure; this can be used when the code no
longer needs the AST.
Manually management by ast:release() can work, but use LRU algorithm to release automatically would be easier and less bug (IMHO) . If we hold some AST for later use, it's safe to release automatically first, and rebuild from code when use again.
