LuaJIT FFI and the FFMPEG library

[Michal Kottman <k0mpjut0r@...>]

Hi,

I wanted to see, how LuaJIT & FFI deal with sound and image
processing, having seen great performance with number crunching. To do
that, you first need to load the data. For the greatest flexibility
w.r.t. various codecs, I wanted to try out the FFI with the FFMPEG
libraries (libavformat, libavcodec). To my surprise, getting the
libraries to Lua using the FFI was easy and quick - it took about 5
minutes along with thinking and preparation :)

All I had to do was to preprocess the headers using gcc.I created a
file tmp.c, in which I wrote several #include lines, and then ran "gcc
-E tmp.c > ffmpeg.h". In Lua, all I have to then do is:

    avcodec = ffi.load('avcodec-52')
    avformat = ffi.load('avformat-52')
    ffi.cdef(assert(io.open("ffmpeg.h")):read('*a'))

And then, "everything-just-works". You only need to know/guess, in
which module each function lies, and call it appropriately (like
avcodec.avcodec_decode_audio3(...)). I spent the next several hours
learning how to decode audio with FFMPEG, which was a PITA because of
lack of good documentation - I had to look up relevant calls in ffmpeg
source files... Fortunately, I've done the work for you, so if you
want to do load audio using LuaJIT and FFMPEG, head over to the
following page and get inspired (consider it public domain):

https://github.com/mkottman/ffi_fun/blob/master/ffmpeg_audio.lua

It loads any file with audio stream that FFMPEG understands, decodes
it, and produces a FFI array of samples for the whole file, which can
be further processed. The process is really fast, a 10MB MP3 file is
processed in ~3 seconds (this involves everything, including library
loading, cdef parsing, decoding and writing the 60MB of raw samples to
disk).