• Subject: Finding the 'double' resolution
• From: Philippe Lhoste <PhiLho@...>
• Date: Thu, 19 Jul 2001 12:43:03 +0200 (MEST)

```I wrote:
> Page 74 (11.4 Queues and Double Queues), you state that indices
> of a double
> queue can grow almost infinitely because they are actually
> doubles, so they
> can be quite large.
> But since the doubles are handled by the compiler / run-time library, I
> suppose they are stored in a limited space (as opposed to large number
> libraries). 48 bits, if I recall correctly (and if this is
> standard, I am not a
> specialist here).
> So after some time, values will be rounded and manipulated with
> exponents of
> ten, so small increments will be lost.
>
> I made a little test:
>
> val1 = 314159265358979323
> val2 = val1 + 1
> print(val1, val2, val2 - val1)
>
> result on Windows NT:
> 3.141592653589793e+017	3.141592653589793e+017	0
>
> OK, I made a small test, the limit seems to be between
> 9,007,190,000,000,000
> and 9,007,200,000,000,000, so you may be quite right finally...
> It may be interesting to state this limit (if it is standard induced)
> somewhere, I think.

I improved my test to better and quicker detect this limit:

-- Find the 'double' resolution
-- ie. the point where an increment of 1 to a Lua numerical value
-- (stored as double C type) is lost.
-- by Philippe Lhoste <PhiLho@GMX.net> http://jove.prohosting.com/~philho/
-- v. 1.0 -- 2001/07/19

--i = 9 000 000 000 000 000
i    = 9000000000000000
incr =  100000000000000
repeat
j = i + 1
if i - j == 0 then
i = i - incr
incr = incr / 10
print('... ' .. i .. ' (' .. incr .. ')')
if incr < 1 then
print('= ' .. i)
break
end
end
i = i + incr
until nil

-- val1 = 9007199254740992
-- val2 = val1 + 1
-- print(val1, val2, val2 - val1)

val1 is the first value, on my system (Windows NT on Duron), where an
increment of 1 isn't signifiant.

Regards.

--
--._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.--
Philippe Lhoste (Paris -- France)
Professional programmer and amateur artist
http://jove.prohosting.com/~philho/
--´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`·._.·´¯`--

Sent through GMX FreeMail - http://www.gmx.net

```