Re: lujit - Malformed number

[Andre Leiradella <andre@...>]

 On 05/09/2010 03:15, Florian Weimer wrote:
> * Rob Kendrick:
>
> > On Sat, Sep 04, 2010 at 10:54:59PM +0200, Florian Weimer wrote:
> >
> > > It could be argued that tostring is lossy for tables, so it can be
> > > lossy for numbers, too.
> > Surely converting a floating point number to a string like this is
> > never not going to be lossy?
> The default conversion in Lua (that is, tostring) is lossy,
> unfortunately.
I'm having exactly the same problem with the serialization of floats and 
doubles to JSON (C++, not Lua.)

If the numbers are not supposed to be hand edited, the %a format 
specifier creates a lossless (depending on the precision used) 
hexadecimal ASCII representation of doubles that can be transformed back 
to double by strtod. It's a one line change in lstrlib.c, function 
str_format:

        case 'e':  case 'E': case 'f':
        case 'g': case 'G': case 'a': { // %a format specifier added
          sprintf(buff, form, (double)luaL_checknumber(L, arg));
          break;
        }

Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> print(string.format('%.13a', 0.1)) -- 13 is large enough to represent 
all significand bits
0x1.999999999999ap-4

Which is 1.999999999999a * 2^-4. And tonumber already converts it back 
to double:

> print(tonumber('0x1.999999999999ap-4'))
0.1

Unfortunately the lexer doesn't like the hexadecimal point but patching 
it must be easy:

> a=0x1.999999999999a0p-4
stdin:1: malformed number near '.999999999999a0p'

Doubles have 52 bits to represent the significand, so 52/4=13 in the %a 
specifier is enough to represent all of its bits. The implicit 1 before 
the significand (0 in denormals) is not part of the fraction so it does 
not count in the 13 digits after the point.

It's a shame I can't use this format in JSON, its specification doesn't 
allow hexadecimal numbers.

Cheers,

Andre