Re: [mildly OT] Some info about Python

[Stefan <ste@...>]

Hey Andre,

Am 15.01.2020 um 22:56 schrieb Andre Leiradella:
> On 15/01/20 21:18, Stefan wrote:
> >  * Fast state creation and destruction even with many C functions
>
> I once toyed with Lua state rollback and reuse, using linear allocators. 
> The idea was to have instant Lua states for use, and fast memory 
> management, for short-lived scripts.

I too tried using Lua with an arena allocator; it speeds up script
execution by about 10-20% since Lua makes heavy use of the allocator.

> The process was like this:
>
> 1. Create a linear allocator
> 2. Create a Lua state using the linear allocator to manage memory (note 
> that the linear allocator does not free memory)
> 3. Check and save the number of bytes used by the Lua state used after 
> initialization, including all necessary modules (state_size)
> 4. Allocate state_size bytes from the allocator and save the pointer 
> (inited_state), and memcpy the initialized Lua state to that new area
> 5. Check and save the number of bytes used in the linear allocator 
> (rollback)
> 6. Use the Lua state to run a script
> 7. Memcpy inited_state over the state at the beginning of the linear 
> allocator
> 8. Set the number of used bytes by the linear allocator to rollback 
> (uncommit the unused pages)
> 9. Go to 6
>
> Of course each linear allocator must have enough space reserved for your 
> worst case, but that's not a problem in 64-bit OSes since the address 
> space is huge.
>
> Cheers,
>
> Andre

Well, that's exactly the problem with area allocators: What if your
data no longer fits into your area?
Abusing virtual memory is extremely hacky.

I haven't yet figured out what exactly takes so long in luaL_openlibs - 
but it is probably the sum of unneccessary object creation/string
interning/hash table construction and collection of the whole shebang
afterwards.

Greetings