lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

You misunderstood the issue: The given number can be represented
precisely by a 64-bit float. In fact all integers up to and including
2^53 can be represented precisely by 64-bit floats. The issue is just
that conversion to string rounds "too much" by default, losing
precision. Thus the round-trip tonumber(tostring(x)) == x fails.

On 20.06.23 13:53, bil til wrote:
I’m running into the following issue (minimal example of a problem in a JSONschema validator):

local maximum = 9007199254740991
Isn't this very obvious that something like this will happen at larger
float numbers? This is typical restriction for float.

Check e. g. Online-converter Float -> hex, then you should see the
problem more clearly (you do not define very well in your post, which
bit resolution you use for your floating point calcualations, this
depends strongly on your machine...).

Due to this, for "counting values" like time or large encoder values,
it is often advisable to use int's.