math.abs(math.mininteger) (Was: Must "attempt to get length of a number value" be an error?

[Dirk Laurie <dirk.laurie@...>]

Op Sa. 11 Mei 2019 om 01:38 het Egor Skriptunoff
<egor.skriptunoff@gmail.com> geskryf:

> According to Lua manual, math.abs() must return the absolute value of its argument.
> Obviously, math.abs(math.mininteger) not equals to the absolute value.
> That's a bug.
>
> "bug" means that behavior does not match documentation.
> Either behavior should be changed (to return correct result)
> or documentation should be changed (to claim math.abs as returning result modulo 2^64 instead of the correct result)

In view of the fact that math.abs(math.mininteger) _can_ be
represented exactly as a lua_Number, I must agree with you.

I tried coding this case. It is suprisingly tricky.

fprintf(stderr,"n=%lli x=%llx  <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);
      if (n<0) n=-n;
fprintf(stderr,"n=%lli x=%llx  <≤=≥>:%i%i%i%i%i\n",n,n,n<0,n<=0,n==0,n>=0,n>0);

produces (GNU compiler):

n=-9223372036854775808 x=8000000000000000  <≤=≥>:11000
n=-9223372036854775808 x=8000000000000000  <≤=≥>:01010

I.e. x=-math.mininteger satisfies x<=0 and x<=0 but not x==0. Curious,
and not to be relied on, according to a 2012 paper on overflow in C
http://www.cs.utah.edu/~regehr/papers/overflow12.pdf which says that
the result of -INT_MIN is undefined in 2-s complement,

The correct coding must test for the edge case and dispose of it
before handing the general case.

      lua_Integer n=ivalue(rb);
      if (n==LUA_MININTEGER) setfltvalue(ra,-(lua_Number)(n))
      else if (n>=0) setivalue(ra,n)
      else setivalue(ra,-n);