[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: Questions about lua_strx2number
- From: Stefan Ginsberg <stefan.ginsberg@...>
- Date: Thu, 19 Mar 2020 17:19:14 +0100
Thank you for your reply.
I will try to find some other value for the decimal case
then since I am trying to use the algorithm of lua_strx2number
but for decimal strings instead of hexadecimal.
> I have some questions about the following macro used by lua_strx2number:
> /* maximum number of significant digits to read (to avoid overflows
> even with single floats) */
> #define MAXSIGDIG 30
> What is the mathematical behind this? That is, why will there be overflow
> with more than exactly 30 hexadecimal digits?
No one said it is exactly. We need a number larger than the number of
significant hexadecimal digits in a float (~14 for double precision, ~29
for some long doubles) and smaller than the number of hexadecimal digits
in the largest representable float (~32 digits for single precision
> Also, is this the case for decimal number strings too or is it another amount?
It would be different for decimal number strings, but we use ISO C'
'strtof/strtod/strtold' for that conversion. (That one is much more
subtle, because of rounding errors with decimal adjustments.)