Re: 64-bit integers (Was: Extending Lua for multiple light userdata types)

[Daurnimator <quae@...>]

On 14 November 2012 11:59, Rena <hyperhacker@gmail.com> wrote:
> I thought it would be appropriate to split this, since the original
> topic wasn't really about 64-bit integers, but an extension to the
> type system, for which implementing those would be one example use.
>
>
> On 2012-11-13 6:41 PM, "Roberto Ierusalimschy" <roberto@inf.puc-rio.br> wrote:
>>
>> > This conversation started in IRC and I fully agree with Rena which
>> > will come as no surprise as I have said this before. It is 2012 and
>> > yet Lua has no access to a 64 bit int(in fact from the C API point of
>> > view it was even considered to remove the default lua_Integer as a
>> > typedef for ptrdiff_t ), before too long the majority of people will
>> > not care about 32 bit systems and even arm is getting a 64 bit
>> > version, when will Rio Lua have a 64 bit int? When will we be able to
>> > sanely do bit operations on 64bit types without resorting using
>> > strings *?
>>
>> I have been playing a little with the idea of introducing an integer
>> type in Lua. I have intended to present some initial thoughts in the
>> workshop.
>>
>> The overall idea is quite simple. There will be two kinds of numbers,
>> integers and floats. All operations except division result in an
>> integer if both operands are integers and in float otherwise. There
>> will be two divisions: float division (the usual '/') always have
>> a float result, integer division ('//'??) always have an integer
>> result.
>>
>> Except for overflows, all operations have the same results
>> independently of whether operands are represented as floats or
>> as integers; but there is no automatic conversion on overflows:
>> operations with integers should overflow like unsigned integers
>> in C.
>>
>> (So, the fact that a number is represented as an integer or as
>> a float is explicit and visible to the programmer, but for most
>> mundane tasks this distinction is irrelevant.)
>>
>> -- Roberto
>>
>>
>
> This all sounds good, I'm just wondering how signed/unsigned would work.
>
> I feel like proper 64-bit integers and bit operators are two things
> Lua really lacks, especially as we move to 64-bit systems. Sure, a
> double can hold a 53-bit integer, and the bit32 library can be used
> for simple stuff, but when you need to do serious work with them, say
> working with binary protocols and file formats, those really start to
> feel like using a coin as a screwdriver.
>
> Take for example, my project to make a map editor for Mario Kart 64.
> To locate the resources in the game binary (which is a single giant
> executable, no filesystem) and decode texture coordinates, it uses
> such lovely equations as:
>
> CmdWord[0] = (ImgType | 0xFD000000) | 0x100000; //Set Texture Location
> CmdWord[1] = (Param[0] << 0xB) + 0x05000000;
> CmdWord[2] = 0xE8000000; //RDP Tile Sync
> CmdWord[3] = 0;
> CmdWord[4] = (((ImgType << 0x15) | 0xF5000000) | 0x100000) | (Param[2]
> & 0xF); //Set Tile
> CmdWord[5] = (((Param[2] & 0xF0) >> 4) << 0x18);
> CmdWord[6] = 0xE6000000; //RDP Load Sync
> CmdWord[7] = 0;
>
> and:
>
> glTexCoord2f(this->VtxCache[Command[i] >> 1].TextureCoordS /
> (float)(32 << this->Tile[0].SMask),
>     this->VtxCache[Command[i] >> 1].TextureCoordT / (float)(32 <<
> this->Tile[0].TMask));
>
> These are ugly enough as it is... replace all the infix binary
> operators with prefix, parenthesised function calls, and it's a
> downright mess. And since the GPU works with 64-bit command packets,
> not having a native 64-bit type creates another complication...
>
> Not every program has to deal with messy bit-packing code decompiled
> from old video games, but plenty of programs deal with similarly ugly
> binary formats, and Lua doesn't really make it easy.
>

Chatting on IRC I think I came to a solution:

Why not just have the single (high level) number type and just have
the underlying type automatically fit the data?
If it has something after decimal place? silently converted to double
Essentially we have multiple number 'sub-types': number.double, number.int64
1 == 1.0 and t[1] is still equivalent to t[1.0]
of course... then 2^60+1 != 2.0^60+1
Anyone could then create more number types (eg, int32, fixed point or bignums)

We'd need a 'number' library:
number.todouble ( 1 )
number.toint64 ( 1.0 )