[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: constant table overflow
- From: David Olofson <david@...>
- Date: Thu, 10 Mar 2005 15:14:54 +0100
On Wednesday 09 March 2005 15.44, Serge Semashko wrote:
> Hello All,
> We are using lua as a storing and manipulating a big set of data
> (arranged as a set of nested tables containing lots of string
> constants). After some time, the size of this data has grown to some
> extent and now we are getting "constant table overflow" error and
> load the file anymore.
> The following script can be used to create a lua file that
> the problem:
> for i = 1, 1000000 do
> print("a = \"" .. i .. "\"")
> And the size of this test file is only 13MB. Maybe such limitations
> are too restrictive? Is lua going to have a bigger constant table in
> the future?
Well, the reason for the limit is that one has to decide on a
reasonable size for instructions. The Lua authors decided on 32
bits/instruction, and with the current layout, that means 8, 9 and 9
bits respectively for the operands - or as in this case, 18 bits,
when B and C are combined into Bx.
It's theoretically easy to "fix", but that comes at a cost: Larger
code and slower execution...
> And is there an easy way to hack lua 5.0.2 to increase this limit?
For linear expansion of the constant indexing range, you could add
LOADK variants that add 2^18, 4^18 etc to Bx before indexing, and
have the coder select LOADK variant as needed to access the requested
Another (slightly more involved) variant would be to add a "long
address prefix" instruction, that loads high index bits that LOADK
then uses to reach an otherwise out-of-range constant. To avoid
having to implement state tracking for this "extended addressing
register" in the coder, you could just have LOADK zeroing those extra
bits after using them, making the "long address prefix" instruction
"one-shot" rather than state changing.
> I don't quite understand constants in 'lopcodes.h' and not sure
> whether it is safe to change them.
There are only 32 bits in an instruction, and there are 35
instructions (in the version I'm looking at), so you can't steal any
bits from the opcode field, at least... You could try making the A
operand smaller, I guess, but then you might run out of addressable
The opcode field has room for around a bunch of new instructions
though (max 64 total), so unless you need *many* times more constants
than the current 2^18, special "high LOADK" instructions might do the
trick. Doesn't look like it would be all that hard to implement,
though it might confuse the optimizer or other parts of the compiler.
//David Olofson - Programmer, Composer, Open Source Advocate
.- Audiality -----------------------------------------------.
| Free/Open Source audio engine for games and multimedia. |
| MIDI, modular synthesis, real time effects, scripting,... |
`-----------------------------------> http://audiality.org -'
--- http://olofson.net --- http://www.reologica.se ---