|
Some operation system, say Windows, use EXTENDED PRECISION by default. (64 bits long double instead of 53 bits double) Example, from my Win7 laptop: Python return the correct rounded sum: (thus *all* Python import modules also does 53-bits float rounding) >>> 1e16 + 2.9999 # good way to test float precision setting 10000000000000002.0 # Python got the rounding right Lua 5.4.0 (work1) Copyright (C) 1994-2018 Lua.org, PUC-Rio > string.format('%.17g\n', 1e16 + 2.9999) 10000000000000004 -- wrong, rounding in long double setting For 53 bits double, to make same math result across platform, maybe add this to Lua.c ? (even if not needed) fesetenv (FE_PC53_ENV); |