Share function proto among multiple lua vms

[云风 Cloud Wu <cloudwu@...>]

Our game server framework skynet [1] use thousands lua VMs in one
process, the function prototype cost large amount memory. So I patched
lua to share function prototype objects among the lua vm in 2014 [2].

As Roberto pointed out, "short strings cannot be shared (at least
directly), due to internalization." [3] . So I copy constants table
(field 'k') for each proto object .

I am glad  that I found a new way to solve this problem these days.

step 1,  I use a thread-safe global hash table to store most of short
strings in prototype , just like Erlang's atom type. (Never delete
them)

step 2, When I clone a proto object, checking the constants table
first. If all the object including short strings can be shared, we
shared the field 'k'. In some rare case, some short string constants
are interning in local vm before entering global atom table, we still
make a copy.

step 3, Turn on the global atom table during loading file.

The results:

Original lua 5.3.5                         : 11419.49 Kb
Share bytecodes                         :  6316.61 Kb
Share bytecodes and constants  : 4614.29 Kb

There are about 2000~3000 vm in one process, so we saved about 1.6M *
2.5K = 4Gb memory. I maintain the patch in my github repo [4] .

I think we can do better in lua 5.4, because lua 5.4 remove the `field
cache` from struct Proto , maybe we can share the whold struct Proto.

[1]  https://github.com/cloudwu/skynet
[2]  http://lua-users.org/lists/lua-l/2014-03/msg00489.html
[3]  http://lua-users.org/lists/lua-l/2014-03/msg00497.html
[4]  https://github.com/ejoy/lua/tree/skynet

-- 
http://blog.codingnow.com