[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: LuaJIT vs MATLAB benchmark [Was: Re: help to (slightly) modify Lua interpreter]
- From: Leo Razoumov <slonik.az@...>
- Date: Fri, 6 Nov 2009 17:25:58 -0500
On 2009-11-06, Mike Pall <email@example.com> wrote:
> Please post current benchmark results vs. Matlab, if you have any.
For whatever it is worth below is a comparison of LuaJIT-1 versus
MATLAB-7.8.0 (February 2009 release) for a scalar tight loop
Comparison is done on 64-bit 4 way Quadcore machine (16 cores total)
* 4 x QuadCore Intel(R) Xeon(R) CPU E5530 @ 2.40GHz
* Linux 2.6.18-128.1.14.el5 #1 SMP Wed Jun 17 06:38:05 EDT 2009
* memory 64GB
* MATLAB Version 188.8.131.527 (R2009a) 64-bit (glnxa64) February 12, 2009
* Lua 5.1.4 (aka LuaJIT-1)
To double check MATLAB results I asked my colleague to run the test on
Windows XP version of MATLAB on Core2Duo laptop @ 2GHz.
The test programs that compute Euler constant are attached to this
message. The results are below. In the case of MATLAB the so called
"Elapsed time" as reported by tic/toc commands is more accurate metric
then output from "time" command. "time" counts Matlab startup time
that can be couple of seconds while MATLAB's "Elapsed time" is pure
script execution time.
* (1) MATLAB (Linux):
N = 1E+07
Elapsed time is 16.714563 seconds.
* (2) LuaJIT-1 (Linux)
N = 1E+07
* (3) MATLAB (Windows XP, different hardware. Core2Duo @ 2GHz)
N = 1E+007
Elapsed time is 12.677472 seconds.
It seems that MATLAB Windows port is better optimized but it is still
about 100 times slower than LuaJIT-1. The joke is that MATLAB has its
own JIT since version 6.5 and JIT was enabled for these tests.
Now you may ask how come that people put up with such a poor
performance of the interpreter. The answer is multi-fold. First they
buy hardware. 16 core machine can simultaneously run 16 independent
and slow simulations and I have an access to a dozen of those boxes.
Often one can convert loops into vector operations and offload them
from the interpreter to high performance numeric libraries included in
MATLAB and these libraries are good! However, if one has conditionals
inside a loop, vectorization becomes problematic. Also it is
difficult-to-impossible to vectorize event driven simulations.
Additionally, vectorization trades off memory for performance. MATLAB
cannot run on an embedded system but LuaJIT can. As the last resort
one can write critical sections in C/C++ using Matlab's MX interface.
There is also a MATLAB compiler but I have not much experience with
LuaJIT performance together with its ability to nicely interface to
native libraries can make it a viable contender in the fields of
numeric and scientific computing.
% Benchmark: Euler constant
% Run as: sh$ time matlab -nosplash -nojvm -r bmark_euler
sum = sum + 1/n;
euler= sum - log(N);
fprintf('N = %G\n', N);
fprintf('euler= %G\n', euler);
local N= 1E7
local sum= 0
for n=1,N do
sum= sum + 1/n
local euler= sum - math.log(N)
print(("N = %G"):format(N))