Re: Lua as a data description language

[Mike Pall <mikelu-0406@...>]

Hi,

Dimitris Papavasiliou wrote:
> although it should be noted that the only problem is with file size and
> hence resource loading time (once you get the data from lua you can cast
> it to whatever type you want so it won't matter how lua stored it).

If you don't care much about loading time then this might work:
- Compress your (compiled) data files with gzip or bzip2.
- Write a chunkreader that autodetects a compressed file and uncompresses
  on-the-fly. The zlib/libbz2 manuals should have an example that does
  exactly that.
- Hook the chunkreader into your own dofile().

I think this will get you excellent compression ratios for the kind of data
you have. And for *all* of it, not just the data you cared enough to
hand-code a compact representation for.

If this does not get you the desired effect, then create a converter script
that reads your 'plain' Lua data files and generates a Lua data file with
a compact representation (using strings and maybe delta-encoding). Then
compile and compress this file. Allow for loading both unconverted and
converted scripts so you don't need to run the converter script all the time
during development.

Ok, so I do see the advantages of this approach for 3d models and other
heavily structured data. But I think you can save yourself some trouble
by using dedicated libraries for e.g. images and sounds (libpng and
Ogg Vorbis come to mind). I guess you won't use anything else than a flat
RGBA or two-channel-16-bit representation anyway. And these libraries give
you exactly that, so why roll your own?

Bye,
     Mike