RE: Converting Lua 5.0 binary chunk/bytecodes to Lua 5.1 binary chunk

[Jaco van der Merwe <jvdmerwe@...>]

Thank you KHMan for the answer. 

The differences of 5.1 relative to 5.0 (from [1]) can be summarised as follows:

* Numeric IDs of some opcodes have changed
* New opcodes have been added
* Some opcodes have been removed
* Some opcodes have been renamed
* Behaviour of some opcodes have changed
* Operand field ordering has changed
* Encoding of operand fields for some opcodes have changed
* Changes to function prototype headers, e.g. is_vararg field
* Changes in ordering, e.g. for function prototype debug data moved to end 

Some of these changes should be fairly trivial to handle. Without knowing too much about the byte codes the following changes seem to be non-trivial:

* Some opcodes have been removed: these will have to be replaced with their equivalents in 5.1. This might be a straightforward 1-to-1 opcode replacement, or may require a transformation into a 5.1 construct that has the same behaviour

* Behaviour of some opcodes have changed: the constructs using these opcodes will have to be transformed into a 5.1 construct that has the same behaviour as in 5.0

@KHMan, since you are the author of ChunkSpy you have much more insight into this problem than I do. I would appreciate if you could comment on the points above.

Regards
J


On 7/10/2012 2:11 PM, KHMan wrote:
> 
> On 7/10/2012 3:53 PM, Jaco van der Merwe wrote:
> > I have a collection of luac files that were compiled with Lua 5.0.2.
> Unfortunately I do not have the original source code files anymore. In an
> attempt to recover the original source code I investigated the available
> Lua decompilers. The only decompiler I found that works for Lua 5.0
> binaries is luadec [1],[2]. It was only able to decompile about 60% of my
> files. On the remaining files it either asserts or complains about
> unsupported constructs in the code. Even the successfully decompiled files
> do not seem 100% correct.
> >
> > The Lua 5.1 decompilers seem to have evolved significantly in terms of
> correctness, robustness and the quality of the generated code, for example
> luadec51 [3]. If I want to utilise these decompilers I'll have to somehow
> convert the Lua 5.0 binaries to Lua 5.1 binaries.
> >
> > My question to the community is whether this has been done before? If
> so, can you provide any pointers to it?
> 
> None IIRC. It's a pretty small niche...
> 
> > If the answer is no, I would appreciate some pointers on how to go about
> implementing such a conversion myself. I have found a few possible
> approaches.
> >
> > * The ChunkSpy [4] utility supports rewriting of chunks from one
> platform format to another. Maybe one can hook into the rewrite function
> to do such a conversion? Some pointers as to the differences between Lua
> 5.0 and 5.1 bytecodes are provided in chapter 15 of the document "A No-
> Frills Introduction to Lua 5.1 VM Instructions" [5].
> 
> That ChunkSpy platform reformatting is just fiddling with some
> data types. Converting binary chunk 5.0 -> 5.1 is certainly
> possible; the best way is to actually do a comparison study. Lemme
> take a quick look...
> 
> > * Another approach might be to base a converter the lundump.c and
> ldump.c files. The implementation would probably consist of modifying the
> lundump.c file from Lua 5.0 to read the chunks and create some
> intermediate data structures. Then one can modify the ldump.c file from
> Lua 5.1 to convert this intermediate representation and write out 5.1
> chunks.
> 
> This would be best, rather than whack at ChunkSpy's spaghetti
> code. A pure Lua script to do this ought to be pretty short too.
> 
> > The approach may seem simple, but I suspect the difficulty will be in
> handling cases where the behaviors of existing opcodes have changed.
> >
> > Regards
> > J
> >
> > [1] http://luadec.luaforge.net/
> > [2] http://files.luaforge.net/releases/luadec/luadec
> > [3] https://github.com/sztupy/luadec51
> > [4] http://chunkspy.luaforge.net/
> > [5]
> http://luaforge.net/docman/83/98/ANoFrillsIntroToLua51VMInstructions.pdf
> 
> --
> Cheers,
> Kein-Hong Man (esq.)
> Kuala Lumpur, Malaysia