[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: [Experiment] Module system based on static hashing
- From: Tim Hill <drtimhill@...>
- Date: Fri, 17 Apr 2020 14:43:32 -0700
> On Apr 8, 2020, at 5:14 PM, Stefan <email@example.com> wrote:
> I've conducted an experiment to speed up Lua state creation with static
> hash tables generated by GNU gperf and got some interesting results.
> Lua states offer a very light-weight way to execute independent scripts,
> which is a much desirable feature for programs that execute a large
> number of them (e.g. web servers).
> Unfortunately the standard library is too small for many tasks and
> adding modules by hand is quite a hassle. Furthermore are dynamic
> libraries a platform-dependent mess.
> The goal of this experiment was to find a way to add many more functions
> to Lua a) without using dynamic loading and b) without slowing down the
> creation of new states.
> luaL_openlibs loads all functions, tables and values such as print,
> string, _VERSION, math.pi etc. that make up the standard library into
> the Lua state so that the script can access them via table lookups.
> But rarely does a script use ALL of them and the more functions get
> added, the more unnecessary work luaL_openli has to do.
> So, the less unused Lua values get loaded into RAM, the better.
> OK, so what if we don't actually load them and just set a metatable
> with a __index metamethod that fetches the values as the script needs
> them? The script won't notice absent values it doesn't use -- Great!
We created a very similar system, with a very similar implementation, a few years back. We were able to reduce the startup times for a new Lua state from 2-3ms to about 50us, which was pretty dramatic (including loading various private libraries). Of course, you later paid a bit extra for the meltable hits when look-up DID occur for library functions, but in our application where the states were short-lived this tax was acceptable.
It’s been running in production now for some time very reliably :)