[Date Prev][Date Next][Thread Prev][Thread Next]
- Subject: Re: Some thoughts on integer exponentiation (and an implementation)
- From: Stefan Ginsberg <stefanginsberg@...>
- Date: Mon, 15 May 2017 00:59:07 +0200
> On 14 May 2017, at 23:46, Sean Conner <firstname.lastname@example.org> wrote:
> It was thus said that the Great Stefan Ginsberg once stated:
>> Something I would find more useful is if overflowed integer results were
>> returned as nil (i.e. 123^456 == nil).
>> Not just for '^' but for all relevant integer operations. This would also
>> make unsigned numbers more manageable.
> Such checks add overhead to Lua (but as usual, one would have to test to
> see of the overhead is unacceptable). Unfortunately, there is no cheap way
> to check for overflow in C (unlike assembly ).
> Second, I'm not sure if I like returning nil.
> a = 1
> b = 63
> c = 1
> x = (1 << b) + c
> stdin:1: attempt to perform arithmetic on a nil value
> stack traceback:
> stdin:1: in main chunk
> [C]: in ?
> Okay, that doesn't happen now---what happens now is that x =
> 9223372036854775807 instead of -9223372036854775809 because of rollover.
> I can see an argument for it, but ...
> -spc (but I'm not sure if I want to see how sloppy I've been with math ... )
>  http://boston.conman.org/2015/09/07.1
> It's not much overhead in time, but it does increase the memory
> usage of the code because of the overhead of checking for overflow.
To be clear that idea is only for the affected mathematical operators, not bitwise shifts.
I don't consider shifting out bits the same thing as overflow, it is another type of operation.
While checked integer math would incur some overhead, compared to something like a call,
table indexing, or even the overhead for doing the operation itself I believe it would be negligible.
For example, checked add:
'if ((ib ^ ic) >= 0 && (ib ^ ia) < 0)'