Re: Integers-related trap in Lua 5.3

[Coda Highland <chighland@...>]

On Wed, May 18, 2016 at 12:00 PM, Sean Conner <sean@conman.org> wrote:
> It was thus said that the Great Roberto Ierusalimschy once stated:
>> > It was thus said that the Great Roberto Ierusalimschy once stated:
>> > >
>> > > One problem with all proposed solutions is math.mininteger. The
>> > > lexer reads numbers without signal. (Several languages do the
>> > > same, e.g. C).  This means that -9223372036854775808 is read as
>> > > -(9223372036854775808) (a unary minus applied over a positive
>> > > constant). But 9223372036854775808 is not a valid integer, so it would
>> > > result in a float (or nil, or inf, or error). Whatever the choice, the
>> > > final value of -9223372036854775808 would not be an integer, despite
>> > > it being a valid integer.
>> >
>> >   I haven't looked at the code, but couldn't the lexer read the minus sign
>> > and set a flag. Then read in the number as an unsigned quantity and as long
>> > as it's not a float (no decimal point, no 'e' or 'E', etc.) then check
>> > against LLONG_MIN and LLONG_MAX? 9223372036854775808 *is* representable as
>> > an unsigned quantity.
>>
>> The lexer cannot distinguish between a "-9223372036854775808" in
>> "x = -9223372036854775808" and in "x = x-9223372036854775808". It
>> handles both the same way, so the only way is a minus and then
>> a constant. (As I said, this is a common thing in programming
>> languages; C and Java, for instance, have this same rule.) There
>> are work arounds, but they are work arounds... Java, for instance,
>> has an axplicit provision for that case:
>
>   There are two cases here:
>
>         x = x-9223372036854775808
>         x = x--9223372036854775808
>
> In the first case, the '-' is the operator and you are trying to subtract a
> positive integer that exceeds the bounds for a positive integer (and then
> "something happens" but that "something happens" is beyond the scope for
> now).  In the second case, you have a '-' as an operator, and a '-' that is
> part of the number (a unary operator).  So you end up with subtracting a
> negative number, which *can* be represented as an integer.
>
>   Or does the lexer not handle this like this at all?
>
>   -spc
>
>

The lexer doesn't handle things like that at all. That's the parser's
job. The lexer just takes a string of characters and outputs a string
of tokens. It could just about be implemented with a ridiculously
large regular expression. (Not that this would be a GOOD idea.) It's
the parser that says "this '-' actually is a unary operator instead of
a binary one, now where's its parameter?"

/s/ Adam