[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
**Subject**: **Re: Questions about lua_strx2number**
**From**: Roberto Ierusalimschy <roberto@...>
**Date**: Thu, 19 Mar 2020 11:01:49 -0300

> I have some questions about the following macro used by lua_strx2number:
>
> /* maximum number of significant digits to read (to avoid overflows
> even with single floats) */
> #define MAXSIGDIG 30
>
> What is the mathematical behind this? That is, why will there be overflow
> with more than exactly 30 hexadecimal digits?
No one said it is exactly. We need a number larger than the number of
significant hexadecimal digits in a float (~14 for double precision, ~29
for some long doubles) and smaller than the number of hexadecimal digits
in the largest representable float (~32 digits for single precision
floats).
> Also, is this the case for decimal number strings too or is it another amount?
It would be different for decimal number strings, but we use ISO C'
'strtof/strtod/strtold' for that conversion. (That one is much more
subtle, because of rounding errors with decimal adjustments.)
-- Roberto