[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: Ann: mydp 0.1 released
- From: David Manura <dm.lua@...>
- Date: Sat, 17 Jan 2009 00:51:50 -0500
On Fri, Jan 16, 2009 at 7:52 PM, Cimarron Taylor wrote:
> 'mydp' is a program for processing mysqldump files in the manner of 'awk'.
> It reads mysqldump sql statements from standard input and executes Lua
> callback functions as table definitions and insert statements are
Maybe slightly more analogous to SAX in XML in that it is a streaming
push parser but with a declared callback function for each item type.
Obviously, you chose this design to support large data files, but it's
flexible. A useful utility function could be written in terms of it
to build an in-memory tree representation (like DOM) or effectively
execute the statements in Lua tables (implementing a rudimentary
in-memory database in Lua). With Lua coroutines, you can invert the
push parser to obtain a pull parser. The serial parsers may also be
chained, as with SAX or according to the filters/sources/sinks/pumps
described in Lua Programming Gems.
Note that if you pass row values individually, "function values(...)",
rather than as a table, you're typically limited to a couple thousand
values. I understand the MySQL limit of columns per table is more or
less the same, so theoretically, this could cause a problem.