Compared to lua 5.2.4, for my usecase I've noticed that 5.3.0 uses roughly 2x the amount of memory and is 1.5x slower. And Lua 5.3.3 is again 2x slower and uses > 4x the amount of memory than 5.3.0.
Do you have test lua 5.3.1 ? I guess the main change between 5.3.1 to 5.3.0 is lua_pushstring (and lua_getfield/lua_getglobal), Lua 5.3.1 add a string cache for C string.
Or you can change all the lua_pushstring to lua_pushlstring in your luabind, and then see the difference.
But I have no idea about 4x memory used, it's strange.