Re: [ANN] Squish 0.2.0 - now with 87.2% more compression!

[Matthew Wild <mwild1@...>]

On 12 October 2010 02:41, David Manura <dm.lua@math2.org> wrote:
> On Sun, Oct 10, 2010 at 12:34 PM, KHMan <keinhong@gmail.com> wrote:
>> Looking at squish_penlight.png, one more good item to compare is "(full
>> minify) (gzip)". This will (I think) compare compressed keywords versus
>> compressed uglified keywords. I've done some testing in the past -- keywords
>> compresses well, what more it's a large file, so the file size might end up
>> quite close to the minimum of 27.6KB, perhaps under 35KB.
>
> uglify's work, i.e. simple Huffman coding, is probably duplicated in
> gzip (Deflate).  Some quick tests here indicate that uglify slightly
> increases file size in the presence of gzip, the main reason probably
> being that doing this embeds two Huffman tables and decoders.  These
> maybe should be merged into a single option to select the compression
> method.
>

I'd be interested in sample files that demonstrate this if you have
them. From my own tests running through uglify strips another 1K from
a squished Penlight and slightly aids gzip decompression time.

> Since embedding the decoder enlarges small files, Squishy might omit
> gzip and print a warning if it doesn't decrease file size.  Perhaps
> there are occasions where a user would still want to force it though,
> such as for consistency or obfuscation reasons, though the latter
> allows other options like AES ( http://luaforge.net/projects/aeslua/
> ).
>

Yes, I would like to do this.

> On Sun, Oct 10, 2010 at 12:52 PM, steve donovan
> <steve.j.donovan@gmail.com> wrote:
>> But of course it would be a very unusual program that managed to
>> _reference_ all of a big library, so the cool thing is that the
>> resulting squished program carries no dead code (or very little). A
>> case where 'static linking' makes a lot of sense.
>
> The static linking thing I've thought would be useful too.  People
> currently do that manually on a small scale via copy/paste.
>
>> It would be a cool idea to assemble a kit which could build a custom
>> Lua interpreter with various necessary libraries like luafilesystem,
>> luasocket, etc built in, depending on configuration.
>
> ...with the help of rockspec metadata.  I think I've mentioned before
> that the squishy file could be partly constructed from rockspec data.
> You could have rock-level (archive) operations like "squish a rock" or
> "statically link two rocks, removing symbols".
>

Ah, I knew someone had suggested that - I wrongly accused Steve in
private and thoroughly confused him :)

I did look into doing this, however rockspecs don't contain enough of
the right kind of information. While they do provide a list of
modules, in the case of a simple module rockspec there is no
indication of which should be the "main" file. I thought about being
smart and basing it on module/filename depth but decided it wasn't
worth the effort right now.

In addition for rockspecs that have Lua scripts in their 'bin' list
Squish still doesn't know which scripts depend on which modules
(without parsing them at any rate, and even then it would just be a
best-guess).

Ideas on exactly how to do either of the above are welcomed.

Regards,
Matthew