Re: Poor md5 module performance

[Luiz Henrique de Figueiredo <lhf@...>]

> Finally I wrote small C program using openssl to calculate this with
> one "shoot".

Reading the file in blocks is fine in Lua if the block size is large enough.
I tried several block sizes with my lmd5 with an added function that does
the whole thing in C (with BUFSIZ blocks). The times reported are computed
with os.clock. The input was a 40Mb file. Of course, doing it in C has the
advantage of not generating garbage in Lua, but that did not seem to make
much difference. The machine I used had BUFSIZ equal to 8192; note how
similar the time is for that block size compared with the C time.

block   time
C        0.16
65536    0.21
32768    0.21
16384    0.20
8192     0.19
4096     0.21
2048     0.24
1024     0.28
512      0.31
256      0.40
128      0.57
64       1.01
32       1.81
16       3.20
8        5.83
4       11.27
2       21.37
1       33.95

Here is (part of) the code

d=D.new()
report("C",d:reset():fupdate(F):digest(),F)
local B=64*1024
while B>=1 do
 assert(io.input(F))
 d:reset()
 while true do
  local c=io.read(B)
  if c==nil then break end
  d:update(c)
 end
 report(B,d:digest(),F)
 B=B/2
end