Re: Performance development Lua 5.1 -> 5.3-rc

[Eric Wing <ewmailing@...>]

On 12/18/14, René Rebe <rene@exactcode.de> wrote:
> Hi all,
>
> did anyone perform some benchmark charting relative to the latest major Lua
> versions, like: 5.1, 5.2, 5.3?
>
> The reason why I ask is that with Lua 5.3-rc0 stuffed into a bigger
> application code base (where the main application is written in Lua) I see
> about 10% performance decrease of the application run-time compared to Lua
> 5.1. It also consumes quite a bit more memory, too.
>
> At this time I have not tested how this app would performs with Lua 5.2.
>
> If anyone has some performance numbers to share I would be intersected to
> hear your upgrade stories. If there is interest in an deeper analysis let me
> know and I can see if I can get the code compatible with all those Lua
> versions and run some automated tests.
>
> 	René



Here's a very small sample point. I did some simple benchmarks for a
brute force sprite benchmark test using Lua language bindings to C (to
SDL). This was with Lua 5.3 work 2, and I didn't do a direct
comparison to 5.1 or 5.2, and I didn't look at memory consumption.

However, I did compare 5.3 to LuaJIT on an iPad mini 1st gen (JIT
disabled). Since I went through normal C/Lua bindings and not FFI,
plus the other constraints, I think the performance between 5.1 and
LuaJIT is virtually identical.


With Lua 5.3 with lua_Number as float, and lua_Integer as 32-bit, I
got indistinguishable results with LuaJIT.

With Lua 5.3 with lua_Number as double, and lua_Integer as 64-bit, I
found it was around 7% slower. (These are the default settings.) Keep
in mind it is a 32-bit processor, so 64-bit sizes are expected to be
more painful.

While my test was mostly C-API heavy, on a desktop with JIT enabled, I
think LuaJIT saw something like a 40% speed up to give you a sense of
how much Lua vs. C bound my test was. (And there should be
little-to-no garbage collection in my test.)


Any benchmarks with Lua 5.3 should take into consideration the size of
lua_Number and lua_Interger and also the architecture you are running
on. I would recommend doing a separate benchmark with 5.3 for both
32-bit and 64-bit sizes. I suspect (hope?) the 32-bit variant comes
closer to your old 5.1 behavior. (Perhaps you should also test a
variant that is double/int.)


Thanks,
Eric
-- 
Beginning iPhone Games Development
http://playcontrol.net/iphonegamebook/