On 6/12/06, Luiz Henrique de Figueiredo <email@example.com> wrote:
0. Our philosophy regarding precompiled scripts is that loading them should be as fast as possible. That goal dictates the current implementation, which only works for native formats. Also, if the bytecode loader is to have a size advantage over the full parser,then the loader must be quite small.1. It is very convenient to precompile scripts once and load them on multiple platforms. The solution is a universal bytecode loader, not a cross-compiler. However, such a loader is bound to be complex and possibly hard to test and maintain, if it caters for many platforms(but that's the whole point).
1. Hmm.. with the existance of both cross compiler, and universal loader, it would be easy to make a regression test suite that finds any bugs introduced automatically. About one days work?
2. The endianness issue. If platforms differ *only* on endianness, then the modified lundump.c that I have posted is the solution. This goes against simplicity of the loader, but the convenience may offset this. Perhaps we could more official and distribute it in etc/.
2. Having the modified lundump.c in etc/ (or a patch) would actually be enough for the fink case I mentioned. Build code could then simply copy etc -> src before building, without needing to generate or fetch the patch from elsewhere. That would do. :) But I still think #ifdef'fed approach is actually better (basically, a question on how official you want to be with the approach).
3. A cross-compiler can be useful if you cannot run luac on the target platform. The solution is a modified ldump.c suited to the target; this will keep the loader as simple as possible, which is the originalgoal.
3. Cross compiler has been requested frequently enough, in my opinion, that I fail to see why luac wouldn't be developed into that direction. Possible in 5.2? :)
4. Lua does not depend on what ldump.c and lundump.c do, as long as the loader builds the correct internal data structures. You can replace ldump.c and lundump.c by anything you want, as long as they agree onthe external format.5. In the current implementation, this format has two levels: a structural level and a physical level. This should make it simple to replace the physical level, which only contains a handful of simple routines. Take ldump.c. The lowest physical level is implemented by DumpBlock and DumpMem. DumpMem is for data that could depend on endianness. So you can, say, write a dumper that saves files in a fixed endianness by simply rewriting DumpMem (which is currently a macro over DumpBlock). The other half of the physical level is implemented by DumpChar, DumpInt, DumpNumber, DumpVector, and DumpString. You can use a different external representation for integers, Lua numbers, and strings by rewriting some of those. The rest of the code in ldump.c implements the structural level. You can use a different structure by rewriting it. The loader is implemented in a similar way. LoadMem is for data that depend on endianness. The swap-aware loader that I posted earlier implements byte swapping in LoadMemand that's the only real change (except for testing the endianness of the file being loaded). Again, you can cater for different external representations by rewriting the correspondingroutines in the loader.Finally, the header of precompiled scripts contain information about the internal format. This can be used to make the necessary decisions for complicaded loaders and bytecode transformers. The header also contains a format number, which can and should be used by anyone writting a differentexternal format.All the modifications discussed above are simple to make, given a specific goal or target platform. It's just that making all of them is too much toinclude in the core Lua distribution.
Perhaps we need someone interested enough on this, to produce and maintain the cross-compiler, and other these issues?
Then again, some parts (the cross compiler, and having endianness read-in support) should at least be covered by the authors. Sorry if we're making your life harder, that's what the users are for!! :) But, they are also fun, aren't they? :P
If you want to modify ldump.c or lundump.c for a specific task, please feel free to contact me (or post here) if you have any questions.--lhf If you have any