Re: Unicode and UTF-8 the Lua way, mid-discussion (was Re: What do you miss most in Lua)

[Tony Finch <dot@...>]

Jay Carlson <nop@nop.com> wrote:
>
> Oh dear, who put that that SJIS fie in that directory?

Veering further off topic, Joey Hess recently blogged about dealing with
mostly-but-not-guaranteed UTF-8 filenames in Haskell:
http://kitenet.net/~joey/blog/entry/more_on_ghc_filename_encodings/

The official standard approach requires applications to either refuse to
handle malformed unicode data or to corrupt it. It seems to me that a
better approach is to treat input and output of unicode data as a
low-level operation that should guarantee that the data can be
round-tripped without corruption. The decision about how to handle
malformed data should occur at a higher level. This approach mirrors what
we have done with ASCII data in the past.
http://fanf.livejournal.com/112378.html

Tony.
-- 
f.anthony.n.finch  <dot@dotat.at>  http://dotat.at/
Forties, Cromarty, Forth: South 5 to 7, occasionally gale 8 except in Forth.
Moderate or rough. Occasional rain. Moderate or good.