Re: Ann: mydp 0.1 released

[David Manura <dm.lua@...>]

On Fri, Jan 16, 2009 at 7:52 PM, Cimarron Taylor wrote:
> 'mydp' is a program for processing mysqldump files in the manner of 'awk'.
> It reads mysqldump sql statements from standard input and executes Lua
> callback functions as table definitions and insert statements are
> encountered.

Maybe slightly more analogous to SAX in XML in that it is a streaming
push parser but with a declared callback function for each item type.
Obviously, you chose this design to support large data files, but it's
flexible.  A useful utility function could be written in terms of it
to build an in-memory tree representation (like DOM) or effectively
execute the statements in Lua tables (implementing a rudimentary
in-memory database in Lua).  With Lua coroutines, you can invert the
push parser to obtain a pull parser.  The serial parsers may also be
chained, as with SAX or according to the filters/sources/sinks/pumps
described in Lua Programming Gems.

Note that if you pass row values individually, "function values(...)",
rather than as a table, you're typically limited to a couple thousand
values.  I understand the MySQL limit of columns per table is more or
less the same, so theoretically, this could cause a problem.