Re: require() parse error in 5.3.4/5.4.0-work2?

[Kenneth Lorber <keni@...>]

> From: Dirk Laurie <dirk.laurie@gmail.com>
> 
>> Are these bugs that should be fixed in Lua?
> 
> No.
> [...]

Ok, good argument for this being a change request and not a bug report.


> From: Stephen Irons <stephen@irons.nz>
> You bring up the point that some functions in the 'standard' Lua 
> libraries check the number and/or type of parameters while others do 
> not.
> 
> * t = require('table', 1234)   -- second parameter is silently ignored
> * n = math.random(1, 10, 100)  -- third parameter causes function to 
> throw an error
> * table.insert(t, 1, 2, 3, 4) -- fourth parameter causes function to 
> throw an error; also if zero or one parameters
> 
> You then ask
> 
> * whether there is a reason for this difference
> * is it a bug in Lua (the language)
> * is it a bug in one or more of the standard Lua library functions (I 
> added this alternative place for a bug)
> * if so, do they need to be fixed
> 
> I suspect that the difference in behaviour is due to implementation, 
> with a bit of history and inertia.
> 
> 'require()' checks that the first parameter is a string (after 
> coertion), but does no other parameter checking. It would take 
> additional code to check for the number of parameters passed in.
> 
> 'math.random()' have three behaviours depending on whether the caller 
> pass zero, one or two parameters. It throws an error (a fourth 
> behaviour) for any more than two parameters. The function is written in 
> C and uses a switch statement on the number of parameters to decide 
> what to do. It is trivial to provide a 'default' case to throw an error 
> for more than two parameters.
> 
> 'table.insert()' has two behaviours depending on whether the caller 
> pass two or three parameters. It throws an error (a third behaviour) 
> for zero, one or more than three parameters. The function is written in 
> C and uses a switch statement on the number of parameters to decide 
> what to do. It is trivial to provide a 'default' case to throw an error 
> for the wrong number of parameters.
> 
> By design, Lua (the language) does not check that a function call 
> provides the correct number or type of parameter. This allows lots of 
> useful things.
> 
> If a function wants to check its parameters, standard Lua operations 
> are available to do so, but each function must do its own checks.
> 
> My conclusion is that there is definitely not a bug in Lua (the 
> language).

Agreed.

> 
> The next question is whether there is a bug in one or more of the 
> library functions.
> 
> * Should 'require()' (and any other function that silently ignores 
> extra parameters) throw an error?
> * Should 'math.random()' (and any other function that throws an error 
> for extra parameters) silently ignore them?
> * Should standard library functions be consistent in the way they 
> handle extra parameters?
> * Should someone put in the effort to make it so?
> 
> There is no easy answer to these questions. The rest that follows is my 
> opinion.
> 
> I would prefer that functions silently accept extra parameters, like 
> Lua (the language).
> 
> I would be happy if 'math.random()' silently accepted 3 or more 
> parameters. It would seem to be trivial to implement, with the 
> 'default:' case in the switch doing the same as 'case 3:', but I 
> suspect my quick look at the code has ignored the size of the stack or 
> something.
> 
> It is slightly more difficult for table.insert(). The error case would 
> have to be restricted to zero and one parameters. The 'default:' case 
> would do the same as 'case 3:', but, once again, I have probably 
> overlooked something.
> 
> In any case, changing the code will mean someone has to spend the time 
> to look at these issues.

I did.  It took about 6 hours for a prototype implementation.  Someone who
knows lua internals better than I do should be able to do it in half that.

> 
> Silently accepting extra parameters might make learning the language or 
> the library functions more difficult. It might make debugging problems 
> more difficult. I have not looked at all places where functions throw 
> an error on extra parameters.

Debugging problems is how I noticed this issue.  A misplaced parenthesis
was very hard to find.

> 
> It is almost certain that if any of the standard library functions 
> changed to the other 'mode', someone's software would break.

I have the luxury of starting a project from scratch, so I can accept
that breakage.  That leaves the question of how many bugs would be found
versus how much actual breakage would occur.  It's an interesting experiment
I will leave to others.

> It is 
> absolutely certain that somebody's expectations would break, because it 
> would be different from before. Someone will complain.

Clearly!

> 
> It is almost certain that someone else will ask the same question about 
> consistency if the standard library functions do not become consistent 
> in the way they handle extra parameters.
> 
> It would be nice on some level if there was consistency in the standard 
> library functions; my preference (which, surprisingly, has not changed 
> over a few paragraphs) would be to silently accept extra parameters, 
> the Lua way.
> 
> Overall, however, I would prefer that the implementors of Lua and the 
> standard library functions continued working on important language 
> issues, rather than fixing minor inconsistencies in already working 
> code.

As a developer, I want all the help I can get.  When C added prototypes,
they broke some working code, added some work for the developer, and found
(and prevented) a LOT of bugs.  When Perl added prototypes, the effect wasn't
as great.  One of the major differences is that having prototypes found lots
of issues when calling C standard library functions - things that are hard to
find without the source to those functions since a debugger can't help.  Perl
has always checked the number of arguments to its built-in functions, so that
situation didn't change.  The case for lua is more like the C case - there is
no good way of detecting this type of error from lua.

(Someone pointed out that I can redefine require to check its arguments before
calling the original require.  That has a tremendous amount of overhead and
is probably harder to write than the C version.  Not to mention as an external
tool you would have to remember to actually use it.  So I don't consider this
a good solution.  Others will certainly have different tradeoffs for their projects.)

I'm not advocating (or even discussing) a general prototype system - that's clearly
a very complex change with some heavy costs involved.

So to put some alpha quality code where my mouth is, there is now a copy
of what I have at the moment at https://github.com/nhkeni/lua/tree/keni-argcount

Thanks,
Keni