Re: Poor md5 module performance

[Marcin Jurczuk <mjurczuk@...>]

On 8 November 2010 01:33, Rob Kendrick <rjek@rjek.com> wrote:
> On Sun, Nov 07, 2010 at 10:44:56PM +0100, Marcin Jurczuk wrote:
>> > return fh and md5.sumhexa(io.open(fname,"rb"):read(10485760)) or nil
>>
>> Try using read("*a").
>>
>> This will read whole file - a those files have > 500MB usually.
>> I need only first 10MB to calculate checksum..
>
> Not interested in corruption or attack on the rest of the file?
>
> The solution here, of course, is to hash a chunk of the file, and then
> skip some of the file using a decision based on the output of the
> previous hash.
>
I need  only first 10MB must be correct - I'm not disigner of this
algorythm .:) This is video stream so if there will be some small
changes in rest of file - it is not a problem..
Finally I wrote small C program using openssl to calculate this with
one "shoot".
Since I have swap on this device I can hadnle one 10MB malloc  and
read with 10MB offset..

Nevertheless chunk size while reading fill is really small in Lua..

-- 
Marcin,