Re: Floats and %d

[Rena <hyperhacker@...>]

On Tue, Jul 21, 2015 at 8:28 AM, Roberto Ierusalimschy
<roberto@inf.puc-rio.br> wrote:
>> > The OP's problem can be solved with something much less ambitious.
>> > Replace the first line above by:
>> >
>> >       case 'd':
>> >           lua_Integer n = (lua_Integer)luaL_checknumber(L, arg);
>> >           addlenmod(form, LUA_INTEGER_FRMLEN);
>> >           nb = sprintf(buff, form, (n);
>> >           break;
>> >       case 'i':
>> >
>> > My question was: which of the other five cases should also be
>> > treated like case 'd?
>> >
>>
>> IIUC, this would break it for some numbers between 2^53 and 2^63
>
> Most of them, actually. It also has undefined behavior for numbers
> larger than 2^63. (On most machines this "undefined behavior" manifests
> as plainly wrong results.)
>
> -- Roberto
>

That can still be addressed:
case 'd':
    lua_Integer n;
    if(lua_isinteger(L, arg)) n = lua_tointeger(L, arg);
    else n = (lua_Integer)luaL_checknumber(L, arg);
    addlenmod(form, LUA_INTEGER_FRMLEN);
    nb = sprintf(buff, form, (n);
    break;
case 'i':

IIRC lua_Number already isn't able to reliably represent values
between 2^53 and 2^63, so the truncation shouldn't be an issue?

I have been using %1.0f instead, but it just feels awkward. %d is
requesting display as an integer, whether the input is an integer,
float, string, etc. I can understand raising an error when information
is being lost, but not in this particular case. If I were concerned
about the precision, I'd use e.g. %1.2f.

Otherwise it seems like the consistent thing would be to raise an
error also for string.format("%1.2f", 0.005), which would be
especially silly.

-- 
Sent from my Game Boy.