lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]


Thank you for your reply.

I will try to find some other value for the decimal case
then since I am trying to use the algorithm of lua_strx2number
but for decimal strings instead of hexadecimal.

On Thu, Mar 19, 2020 at 3:02 PM Roberto Ierusalimschy <roberto@inf.puc-rio.br> wrote:
> I have some questions about the following macro used by lua_strx2number:
>
> /* maximum number of significant digits to read (to avoid overflows
>    even with single floats) */
> #define MAXSIGDIG 30
>
> What is the mathematical behind this? That is, why will there be overflow
> with more than exactly 30 hexadecimal digits?

No one said it is exactly. We need a number larger than the number of
significant hexadecimal digits in a float (~14 for double precision, ~29
for some long doubles) and smaller than the number of hexadecimal digits
in the largest representable float (~32 digits for single precision
floats).


> Also, is this the case for decimal number strings too or is it another amount?

It would be different for decimal number strings, but we use ISO C'
'strtof/strtod/strtold' for that conversion. (That one is much more
subtle, because of rounding errors with decimal adjustments.)

-- Roberto