Re: Lua bytecode doesn't compress well (with LZMA)

[KHMan <keinhong@...>]

Juris Kalnins wrote:
> > Squish actually uses this as its main output filter,  (there's more
> > than one, such as the one that replaces Lua keywords with single bytes
> > > 128) - and it works really well.
>
> And it's very simple to modify llex.c to parse such alternative tokens,
> so that reduced scripts can be used directly (btw, Lua also has 20+ unused
> bytes from the 0..31 control character range).

I have one query for Matthew, the author of squish. Want to find 
out (the lazy way) a few data points (always love data points) to 
get a feel of things:

Does anyone have a set of data points for squish, where we compare 
data-compressed sizes of sources, where:
(1) keyword token replacement filter, versus
(2) no keyword token replacement

Now, in (2), LZ coding would zap most keywords into a sliding 
dictionary match code, whereas in (1), the initial size of the 
filtered sources will be smaller but there is more variation in 
the symbol frequencies of the source code (token symbols added) 
and less chance to make sliding dictionary matches.

So, would there be a big difference when we compare compressed 
sources? Say, we tabulate results as:
(1) original
(2) token filtered
(3) original, compressed
(4) token filtered, compressed

--
Cheers,
Kein-Hong Man (esq.)
Kuala Lumpur, Malaysia