Re: How to calculate MD5 of huge files?

[Jerome Vuarand <jerome.vuarand@...>]

2010/4/22 "J.Jørgen von Bargen" <jjvb.primus@gmx.de>:
> Hi folks.
> I'm using LuaForWindows (migrating from perl for various reasons) and I
> wonder, how to calculate MD5 over huge files (e.g. ISO-Files).
>
> In perl I can add files via a handle to a sum (excuse the use of a foreign
> language in this group ;-)
>
>   $MD5->reset;
>   open(FILE,$file) or die "open($file) failed($!)\n";
>   binmode(FILE);
>   $MD5->addfile(*FILE);
>   close(FILE);
>   my $hex=$MD5->hexdigest;
>
> In lua I've only found the module md5, which provides
>
>    md5.sumhexa(message)
>
> but this requires reading the whole message into memory, which is quite
> impossible for 7GB large files.
> Is there any other solution?

You can use LuaCrypto :

require 'crypto'

local evp = crypto.evp.new('md5')

local file = io.open(... or "bigfile.iso", "rb")
while true do
	local chunk = file:read(16*1024) -- 16kB at a time
	if not chunk then break end
	evp:update(chunk)
end

print(evp:digest())