Re: Conversion of data-file contents into database using Lua

[Petite Abeille <petite.abeille@...>]

On Jan 15, 2012, at 9:14 AM, Sandeep Ghai wrote:

> I am using Lua programming to make a script that helps in converting my
> data-file directly into database tables'.

FWIW, bellow are a couple of scripts that convert all the IMdb files [1] into  a database structure [2].

http://dev.alt.textdrive.com/browser/IMDB/IMDB.lua#L102

The general setup is as follow:

{ iterator = 'Movie', file = 'movies.list.gz', table = 'movie' }

(1) For each source file, there is iterator (reader)
(2) The content of the iterator is written to a given database table

The first step is handled by dedicated readers, as each source file is different in structure and content.
The second step is handled by a generic ETL module (extract, transform, load [3]), which uses the database data dictionary (metadata) to properly populate the target table and resolve all its relational constraints.

http://dev.alt.textdrive.com/browser/IMDB/ETL.lua#L230

> Does Lua handle this?

Just a small matter of programming [4].


[1] http://www.imdb.com/interfaces#plain
[2] http://dev.alt.textdrive.com/browser/IMDB/IMDB.ddl
[3] http://en.wikipedia.org/wiki/Extract,_transform,_load
[4] http://en.wikipedia.org/wiki/Small_matter_of_programming