Re: Could Lua itself become UTF8-aware?

[Gé Weijers <ge@...>]

Roberto,

You're not very far off the mark.

On Mon, May 1, 2017 at 6:00 AM, Roberto Ierusalimschy <roberto@inf.puc-rio.br> wrote:

However, the Lua lexer should not depend on libraries. It would be great
if the lexer could handle Unicode (correctly 

!), but I don't know how to
do it without at least doubling the size of Lua. To do it (very) broken,
I prefer to keep it as it is.

 

Russ Cox' RE2 regular _expression_ library, which supports Unicode, is 50% larger than the Lua interpreter, after comments are stripped. Tables take about 6200 lines of that. I'm sure those tables could be represented in a more compact way, but those tables still have to be interpreted, so the Lua interpreter would grow very substantially.

Anyone who wants to do it the right way should probably read this first:

  http://www.unicode.org/reports/tr31/

Gé