Re: Wanted: Lua source code cruncher

[Chris Marrin <chris@...>]

On Mar 9, 2008, at 12:10 PM, KHMan wrote:

> Chris Marrin wrote:
> > On Mar 9, 2008, at 8:20 AM, David Given wrote:
> > [snip]
> > > > > With C, I can drastically reduce the size of the source code by  
> > > > > using a
> > > > > cruncher utility: this reads in the C source and emits an  
> > > > > equivalent
> > > > > but
> > > > > smaller source file by removing whitespace, comments, renaming
> > > > > locals to
> > > > > smaller versions, etc.
> > > >
> > > > My lstrip does all that, except rename locals; but it's in C.
> > > [snip]
> >
> > I'm not sure of the nature of your app, but maybe it would be  
> > easier to
> > just compress your Lua source using gzip and then use gzio or  
> > something
> > to uncompress it? Gzip does a great job of compressing things like  
> > long
> > variable names which repeat throughout the text. You may still want  
> > to
> > get rid of comments to get maximum compression, though.
>
> For source code, comment and whitespace removal is a pretty big
> win. To improve further by renaming locals will help less, but
> AFAIK nobody's written a tool to do that yet. I think it is a
> limited 'win'. To illustrate, say we have two versions:
>
> A:	i = i + 1
> 	i = i + 1
>
> B:	foobar = foobar + 1
> 	foobar = foobar + 1
>
> B is longer by 20 bytes. For version A, 'i' will be one literal
> code, the second 'i' will be one literal code, and the entire line
> 2 will be one match code. For version B, 'foobar' will be
> literals, the 'foobar ' will be one match code, and line 2 will
> still be one match code. Roughly, compressed B is longer than
> compressed A by 4 literals and 1 match, say about 6 bytes in
> total, less with compressed literals. I suspect the overall effect
> of renaming locals for compressed source code is just a few percent.


But it's worse than that. Changing variable names in a language like  
Lua is not really possible, because of runtime property lookup.  
Imagine this:

    local a = { }
    a.myLongPropertyName = 10
    local b = a["myLongPropertyName"]

That might not be the best style, but it is legal. And if you changed  
the property name, the above would fail, unless you changed the string  
to match. You MIGHT be able to get a preprocessor to catch the above  
case, but you couldn't catch them all. Consider this:

function getBase() return "my" end
...
local b = a[base().."LongPropertyName"]

finding all variants of the above is an impossible task.

-----
~Chris
chris@marrin.com