Re: Of Unicode in the next Lua version

[Patrick Donnelly <batrick@...>]

On Sun, Jun 16, 2013 at 7:30 AM, David Heiko Kolf <david@dkolf.de> wrote:
> The last point, the non-canonical UTF-8 encodings, is actually a huge
> security risk that already opened holes in the field.
>
> UTF-8 is quite often used as just an extension to ASCII (which it was
> meant to be) and so some filters checked that URLs don't use "../" to
> access upper directories.  They did not check all the non-canonical ways
> of encoding dots and slashes so these paths went through the filter.  At
> some point (I guess just before the OS API) the UTF-8 was converted
> "forgivingly" to UTF-16 and suddenly the dangerous paths were used.
>
> That is the reason why the standards say that the conversion of UTF-8 to
> codepoints must not tolerate non-canonical encodings but either reject
> the string completely or put some codepoint in there that signals an
> encoding error (though I do not know which codepoint that was).

Coincidentally, Spotify had problems with this reported in a blog post
today [1].

[1] http://labs.spotify.com/2013/06/18/creative-usernames/

--
Patrick Donnelly