Numeric for loop in Lua 5.3, thoughts reframed

[KHMan <keinhong@...>]

Hi all,

TL,DR
=====

Although Lua 5.3 has largely separate functionality for integer 
and float numeric 'for' loops, mixing integers and floats in 
setting up numeric 'for' loops should be avoided even though 
mixing is allowed for VM completeness. In this respect, I think 
the idiom "for i = 1, math.huge do" should be avoided.

Body Text
=========

(I apologise if my previous communication(s) on this have been 
inadequate. Here is a more coherent treatment.)

'for' loop corner cases (integer):

-- print biggest 8 integers on Lua 5.3.4
for i = 0x7FFFFFFFFFFFFFF8,0x7FFFFFFFFFFFFFFF do
  print(i)
end

Of course, one can say that this also breaks in C, the behaviour 
is expected, exactly as intended. In C, one has the option of 
switching to unsigned long long. If the '<' test is used in C, 
then the out-of-range condition is more obvious. Or use a separate 
variable. For corner cases such as these, sometimes a 'while' loop 
is better. In practice, the vast majority of code would not go 
near corner cases. But generally we avoid testing for the 
0x7FFFFFFFFFFFFFFF limit because it is broken for 'for' loop 
operation for (signed) long long.

The "for i = 1, math.huge do" idiom:

I have seen this idiom used on the list. In my copy of PiL4, 
logical page 60, it says:

    ... If we want a loop without an upper limit, we
    can use the constant math.huge:

        for i = 1, math.huge do
        ...

It is also used elsewhere in the book. Initially I thought this is 
a cute idiom. But what happens when we mix integers and floats? 
For the purposes of this discussion, let us consider the positive 
case only. Now, the reference manual does not say anything about 
how mixed-number loops are set up. But in Lua 5.3.4 
lvm.c:forlimit(), a number that is too large (math.huge in the 
example) causes LUA_MAXINTEGER to be chosen as the loop limit.

Assume for the example that LUA_MAXINTEGER is LLONG_MAX, or 
0x7FFFFFFFFFFFFFFF. So the integer numeric 'for' loop runs, runs, 
then wraps after 0x7FFFFFFFFFFFFFFF. PiL4 says "a loop without an 
upper limit", yet it wraps and pumps out negative numbers. The 
expectation of the coder may be "a loop without an upper limit", 
but it appears more like a 2s complement infinite loop which 
includes negative numbers. Does the expectation of the coder match 
the actual operation of the loop?

Perhaps we should let this slide since integers are 64 bit in 
size. The limit is so far away. But add LUA_32BITS to your 
Makefile and the loop without an upper limit now runs up to 
0x7FFFFFFF, then it wraps and go on forever. It can be reached 
quickly on a modern CPU. In C, we want to code loops in the most 
unambiguous manner possible, but here is a limit test that do not 
actually work, helpfully substituted in by the Lua VM.

True, 2s complement arithmetic and looping is behaving exactly as 
it should. But mixed numbers and out of range conditions in 'for' 
loops is not discussed in the reference manual. If this is 
precisely the intended behaviour, shouldn't potentially surprising 
things be noted in the reference manual?

    for i = 1, math.huge do

Is this a good idiom? Should math.huge be encouraged as an 
acceptable limit value in order to simplify 'for' loops? Should 
this be a substitute for infinite loops?

What is the expectation of the coder? How does the coder expect 
the loop to behave? What really happens when the loop runs? Is 
this acceptable in production quality code?

    for i = 1.0, math.huge do

Here is an all-float version. How does this behave? How does this 
behave using a LUA_32BITS-compiled executable? Do you really want 
math.huge here either?

I like loops with zero surprises. There are potential pitfalls in 
mixing integers and floats in 'for' loops. There are potential 
pitfalls in trusting the programming language to understand what 
you intend to do when putting something like math.huge as the loop 
limit.

--
Cheers,
Kein-Hong Man (esq.)
Selangor, Malaysia