Re: [ANN] Lua 5.2.0 (alpha) now available

[David Manura <dm.lua@...>]

On Fri, Dec 3, 2010 at 12:09 PM, KHMan <keinhong@gmail.com> wrote:
> On 12/4/2010 12:50 AM, Roberto Ierusalimschy wrote:
>>>The manual seems to have missed mentioning
>>> hex floating-point literals in Section 3.1.
>>... So, hex floating-point literals is a kind of "bonus" feature that
>> you get when using a C99 compiler.
>...IMHO if it's enabled in the lexer, it deserves to be mentioned.

Should the lexer be allowed to accept tokens that aren't specified in
the reference manual?  Should the lexer be allowed to reject tokens
that are specified in the reference manual (e.g. name too long)?

One example where not well specifying the lexer could be a concern is
an eval built from untrusted user input [1].  Reliably of lexer
reimplementations could also suffer.

I thought the direction in 5.2 was to make the lexer invariant to the
system, e.g. the locale [2], and certainly more portable than
bytecodes.

BTW, a related question is the specified treatment of unrecognized
string escape sequences [3].

[1] http://en.wikipedia.org/wiki/Eval
[2] http://lua-users.org/lists/lua-l/2009-11/msg00999.html
[3] http://lua-users.org/lists/lua-l/2010-10/msg00449.html