Re: Large number bytecode weirdness on different platforms

[Rici Lake <lua@...>]

On 29-Jan-07, at 4:40 PM, Chris wrote:

<snip>

> As you mentioned it looks like the Windows version is overflowing.
> Ah, now I see it's a problem with the "0x" hex parsing because if I
> use the normal integer value there is no problem.  Ugh, I hate
> inconsistent behaviour ;)

OK, my guess was off by 2^32-1. But I was close, in a sense :)


> Thanks!  I'll have to remember "-l" next time.  I'll look into the 0x
> parsing and see what the issue is.

Lua uses strtod() to "parse" numbers. (Plus some hackery to work around 
the locale issue with strtod().) If that doesn't work, and the number 
looks like it might be hexadecimal (i.e. it has an 'x' or an 'X' in 
it), then it tries strtoul(). If strtoul() is given a number out of 
range, it returns ULONG_MAX which is 2^32-1 if long is 32 bits.

Ubuntu has a strtod() which understands hexadecimal floating point 
numbers, so the strtod() conversion works. Windows doesn't, so Lua 
falls back on strtoul(). It probably should check to see if the result 
in ULONG_MAX and errno has been set to ERANGE, although I'm not sure 
that the setting of errno is done consistenly by different libc 
implementations anyway. But, it doesn't, so that's what you end up 
with.

Unfortunately, even if you have a libc which implements the C99 version 
of strtod() (and consequently would handle hexadecimal floating point), 
Lua won't let you use hex floats in source code unless the exponent is 
unsigned. (The lexer doesn't recognize the 'p'/'P' as an exponent.) 
However, tonumber() will work on hex floats if strtod() works on hex 
floats. Consequently, you can use tonumber"0x2a3p-7" (at least on 
FreeBSD and Linux, I haven't checked it too extensively.)

rlake@freeb:~$ lua511
Lua 5.1.1  Copyright (C) 1994-2006 Lua.org, PUC-Rio
> = tonumber"0x2a3p-7"
5.2734375
> return 0x2a3p-7
stdin:1: malformed number near '0x2a3p'

Note that the value printed for 0x2a3p-7 is exact, which is why I think 
hex floats are useful: they allow you to exchange floating point 
numbers between systems without having to worry about differences in 
the implementation of strtod() which might affect accuracy.